Ferguson, A; Lee, E --- "Desperately seeking … relevant assessment? A case study on the potential for using online simulated group based learning to create sustainable assessment practices" [2013] LegEdDig 33; (2013) 21(3) Legal Education Digest 5

---

Desperately seeking ... relevant assessment? A case study on the potential for using online simulated group based learning to create sustainable assessment practices

A Ferguson and E Lee

Legal Education Review, Vol 22, No.1, 2012, pp 121-145

Over the last couple of decades in Australia a number of factors have set the scene for a radical rethinking of the way practical legal training should occur between the completion of an undergraduate degree and admission to legal practice.

Firstly, there was a shift from practical legal training on the job, based on articles of clerkship (‘Articles’), to practical legal training within a tertiary education setting. Secondly, the growth of the internet has led to the creation of a market imperative that online practical legal training is provided. Thirdly, the student population has developed socially and economically in such a way that it is now demanding flexible, but relevant, legal education that will accommodate full-time work, families and other social and community commitments. Finally, the shift to institution-based rather than workplace-based practical legal training has led to the development of competencies as the basis for assessing whether students are ‘ready’ for practice.

Since January 2010 the Australian National University Legal Workshop’s response has been to provide practical legal training by placing students into ‘firms’ of ‘junior solicitors’ to carry out legal transactions in an online simulated legal environment.

As an adjunct to embracing this simulated legal environment and enforced group work, the purpose and identity of assessment has become less focused on examining a student in a summative sense to check for competence. It has, rather, taken on a drifting, less tangible (but no less significant) role that is more akin to the ‘formative-plus-student self-assessment’ model for assessment espoused by Boud and Falchikov.

In summary, there is a general feeling by the law student population that, to meet its needs, legal education and assessment in legal education needs to be responsive and relevant to a diverse legal profession. And, broadly speaking, current methods of teaching are not meeting this need.

Furthermore, writers such as Boud have taken the discussion of formative learning and assessment and extended its relevance beyond academia to espouse the need for an assessment regime that functions as the catalyst for students to become active players in assessment rather than passive receivers of marks or feedback so that they can ultimately develop the lifelong self- assessment skills to make complex judgements about their own work.

The evolution of a significant change in educational direction by any institution is never simple. As the purpose of this article is not to do justice to or describe the mechanics, trial and trauma of this change in depth, it is sufficient to indicate that the work of Boud and Falchikov sparked an interest in changing the strategic direction at ANU Legal Workshop to achieve sustainable assessment and lifelong learning. Once this occurred, a search began for an educational framework to meet this need within the constraints of a competency and institutional based online environment. A further encounter with Paul Maharg and the SIMPLE project team at Strathclyde University provided a new pedagogical direction that ticked many of the required boxes needed to create sustainable assessment. SIMPLE was described as a:

SIMulated Professional Learning Environment [that] enables students to engage in online simulations of professional practice. Its special pedagogy is based on transactional learning: active learning through performance in authentic transactions involving reflection in & on learning, deep collaborative learning, and holistic or process learning, with relevant professional assessment that includes ethical standards.

Some of the other mechanisms enabled by SIMPLE were for students to receive feed-forward and feedback on work in an ongoing fashion and the opportunity to redo work until such time as it had reached a competent standard. In addition, the SIMPLE environment enables students to work in groups to complete tasks; exposes them to legal content areas in context; and presents both content areas and assessment in an integrated fashion rather than as silo subject areas or assessment timetables.

The result of this fusion of Boud/Falchikov and SIMPLE was the development of an 18-week, entirely online, professional practice core course. This course, named the Professional Practice Core (PPC) Course, integrated the previously individually taught areas of property, commercial litigation, civil litigation, trust accounting and ethics as well as the barely taught practice management competencies into a single course environment. This integrated learning environment (ILE) engages students simultaneously on two levels: (1) An ‘in role’ simulation environment comprised of a virtual office space (VOS) where students work in groups with up to four other students to create a team of lawyers engaging ‘in role’ in transactions in a virtual firm environment. In this space students work to complete transactions involved in conducting files with direction and support in the form of task allocation, and receive feed-forward and feedback from Senior Partners, Associates, clients and other characters that are representative of a lawyer’s experience in practice (‘in role’ learning); and (2) An ‘out of role’ support website using the university’s learning management system allows students to engage with each other and teaching staff to access and process additional resources and information to support their ‘in role’ learning. In this area students are also required to complete individual assessment tasks (‘out of role’ learning).

Assessment within this environment is integrated within the tasks, so that it is occurring in an ongoing manner and involves substantial formative feedback and feed-forward. Students must operate as a group to discern the tasks, problems and issues that need to be addressed and then provide and resubmit their responses to their supervisors until such time as they reach a competent standard.

In addition to the VOS used for the ‘in role’ work, an ‘out of role’ learning website is also provided. This site is a backup to student learning and offers: a repository of helpful links and resources; an ‘out of role’ Q & A and discussion forum; and information about and the location for all individual ‘out of role’ assessment tasks.

‘Out of role’ assessment takes the form of formative quizzes, reflective writing pieces, individual transactional tasks (such as conducting an interlocutory application by web conferencing or the completion of a set of legal practice trust accounting books). Importantly, ‘out of role’ assessment is not seen as more worthy, or as an alternative to ‘in role’ assessment. Rather, it is designed to complement the work being done in the ‘in role’ VOS of the student firm group.

The design of the ‘out of role’ assessment is aimed at motivating individual learning and achievement, and also rewarding those who have actively engaged in the group work. For example, if a student has not engaged with the group work, they will find it far more difficult to complete their individual tasks. Thus, while the individual assessments are designed to enhance and complement the group learning experience, there is an individual warranting function built in as well – we can warrant that individual students have achieved a competent rating against the APLEC admitting requirements without ‘riding on the coattails’ of their peers in the group work.

In addition, assessment is only graded as being: Not Yet Competent (NYC), Competent (C) or Higher Level Performance (HLP) and does not involve numerical marks or any form of bell curving. Students must reach at least a competent standard on all group and individual work in order to successfully complete the course; and neither group nor individual work is rated higher than the other.

The end result of the combined regime of ‘in role’ and ‘out of role’ assessment tasks is that students are exposed to a plethora of ‘assessment’ formats in an ongoing fashion throughout the This mixture of assessment styles allows assessment to appear as both summative (by allowing us to externally warrant that students have demonstrated at least a basis competency) and formative (the assessment is so integrated within the required tasks that it is central to the learning process).

At ANU Legal Workshop, the adoption of the SIMPLE based model for the Professional Practice Core Course was essentially aimed at bringing back a more practical, participation-based learning system.

Boud and Falchikov have suggested that a sustainable assessment regime could include a combination of the following characteristics: (1) engages with standards and criteria and problem analysis; (2) emphasises importance of context; (3) involves working in association with others and authentic representations and productions and promotes seeking appropriate feedback; (4) fosters reflexivity and considers risk and confidence of judgement; (5) builds learner agency and constructs active learners; and (6) Requires portrayal of outcomes for different purposes.

By analysing the case study of the PPC course and assessment structure against each of these characteristics, we may gain some guidance as to whether a sustainable assessment regime has been/is being created.

Within the ‘in role’ learning environment students do not receive tasks according to a timetable with set criteria next to them. Instead they receive letters of instructions from clients or videos of a client interview or an internal memorandum with accompanying authentic documents from their Senior Partner. These outline an issue or give a short description of the legal task required. This means students are engaged in identifying, from the information presented to them, the parameters, structure, form and deadlines for the tasks received. They then need to do the appropriate research individually and as a group to scaffold their knowledge in order to complete the tasks. Options for appropriate assistance in scaffolding their knowledge are provided through: (1) their practice mentor (a legal practitioner assigned to each firm to assist with mentoring the group dynamics and practice management and professionalism skills); (2) the ‘in role’ characters such as associates, clerks and senior partners (‘played’ by a legal practitioner assigned to the firm) and fellow group members; (3) the ‘out of role’ discussion forums and resource library; and (4) external sources of information such as the internet, previous undergraduate course notes and texts and (if they work in a legal environment) their own work colleagues.

Furthermore, within the group and over the course of their interactions with clients and

Senior Partners/Associates, the students need to develop the skill of identifying the criteria and standards for a competent piece of work. Broad criteria are available to students in the ‘out of role’ students’ space regarding the nature of appropriate professional writing styles and analysis. However, students must engage with the work provided in good faith and thus learn the standards to be expected by the responses they receive in the feedback and feed forward.

In a practical legal training environment, context is fundamental. Students need to learn to join together their knowledge from undergraduate studies with the relevant skills required to make a competent and professional lawyer, such as effectively communicating, researching, advising, negotiating, and advocating. By requiring students to work in groups within a simulated law firm environment to complete entire matter files, they must learn to locate issues in the context of the ‘messy’ legal file and the functioning legal business.

In addition, the group of students must assess among themselves what aspects of their work require feedback from each other or from the Associate or Senior Partner. Ultimately, a confident team of students may reach a point near the end of the course where they do not race to meet the optional deadline of providing work to the Associate for feed-forward because they feel that their work is of a standard to go directly to the Senior Partner for sign off.

Furthermore, as a significant number of the ‘out of role’ individual assessments are extensions of the simulated legal context the students are experiencing in the ‘in role’ group work, even the individual assessments are not removed from the ‘realities’ of legal practice.

As a starting point, the students are required to work with the same student group of four for the group work and assessment within the ‘in role’ Virtual Office Space for the entire 18 weeks of the course. As the group is required to submit only one combined response to any task within the virtual office environment, they must necessarily work together to combine resources, draft, edit and re-edit each other’s work in order to complete the matter file. As a result, the egalitarian transference of knowledge and assisting each other to discuss solutions to issues is actively encouraged. Importantly, this environment requires them to demonstrate and develop the essential skills of how to give and receive feedback on all work from their immediate peers in an ongoing fashion.

Student groups are also ‘paired off against’ each other on either side of the matter files they are dealing with. As such, the student groups must negotiate how to conduct a matter when there are ‘real’ solicitors acting for the other side and, as such, they must deal with situations where the solicitors may not act as they had hoped – not returning work on time, not advising their clients as expected, or not seeing the issues as they see them.

In addition to working with other students, the student group and individuals must work in association with the other third party ‘characters’ within the virtual office environment –clients, clerks, Associates, banks, Senior Partners, administrative staff – to ascertain and complete the tasks required to complete the matter file to a competent standard. These ‘characters’ are ‘played’ by real, often currently practising legal practitioners, who are casually employed and trained to play the roles.

Furthermore, the students can engage with lecturers and employed legal practitioners within the ‘out of role’ space to further their understanding of concepts and the practice of law. Most significantly, the students engage with their practice mentors (legal practitioners employed to mentor the firm groups) to develop an understanding of the legal practice context and what constitutes fundamental elements of practice management.

The integrated, authentic PPC simulations require students to make linkages between: (1) the content knowledge obtained in their undergraduate degrees and private work experience; (2) the resources available within the ‘in role’ and ‘out of role’ spaces; and (3) their independent research and the knowledge and experience of the people they are working with, to construct solutions to the problems they have identified both within the group ‘in role’ space and when completing their individual assessment tasks.

For example, within one of the commercial transactions, the students had to negotiate a confusing web of interconnected directors and trustees duties to negotiate settlement of a conflict that had arisen between these duties in a manner that suited all parties (who wished to remain friends and business partners). Some students took a very litigious approach to the situation but then realised in the end of transaction debrief with their mentor that, by doing so, they had failed to actually meet their client’s goal of continuing to work together. As long as the students who took the litigious approach were able to recognise that other pathways may have been more successful in providing a solution to the client’s needs, they could still achieve competency for this area of practice.

As such, the simulated transactions encourage students to test their ‘old’ theoretical knowledge in a practical environment and then self-assess whether they have been successful in doing so within an authentic legal practice context.

While it is difficult to suggest that the PPC Course actually allows students to create their own assessment tasks in order to build learning agency, it is arguable that a number of features of the PPC Course assist in building learning agency and constructing active learners.

Firstly, the need for students to discern the required tasks from a matter file rather than simply doing the tasks outlined on an assessment timetable means that they do effectively have to create an assessable task out of the material provided and then construct solutions based on the resources and knowledge available to them.

Secondly, the fact that students have to continue to do work by responding to the feed-forward and feedback that they receive from fellow students and their Associate until they get it right is also motivation to be fully engaged with the work in the first place. For example, as there is no ability to just choose the best pieces of assessment to be included, or only engage with ‘the most significant’, or easiest pieces of assessment in order to ‘merely pass’, students are motivated to be engaged with the task from the beginning.

This motivation is further encouraged through the advertisement of the clear link between participation within the group work and the individual assessment, and by the fact that there is also a very evident link between the work required of them and the work they will need to complete upon entering the profession.

With the focus of the PPC Course on providing opportunities to develop exactly those skills required to complete the tasks of a legal practitioner, it is definitely arguable that the course ‘identifiably leaves[s] students better equipped to complete future tasks’. Indeed, students themselves have commented in the end of course evaluations that they feel better equipped to deal with going into practice now that they have had the opportunity to practise skills that ostensibly appear to be applicable to the profession they are seeking to enter.

In summary, assessment within the PPC is beginning to work in harmony with the innovations in curriculum development and delivery. At times, assessment may be hidden altogether; and the assessors, instead of being arbitrary markers, become mentors and role models for the legal profession. As such, they are able to use assessment to assist in creating competence for the profession whilst also completing the necessary warranting function required for entry level lawyers. It may just be an example of sustainable assessment in action.