[Home] [Help] [Databases] [WorldLII] [Feedback]
	Murdoch University Electronic Journal of Law

Co-operative Student Learning in Undergraduate Law: Fostering Teamwork Skills in External Students

Author:	Anne Matthew LLB Associate Lecturer, Queensland University of Technology School of Law
Issue:	Volume 10, Number 2 (June 2003)

Contents:

• Introduction
• The development of an assessment model designed to foster teamwork skills in external students
• Development of the Assessment Model
• Review of current and alternative approaches
• The Impact of Technology on Distance learning
• Learning in Internet-facilitated Virtual teams
• Analysis of difficulties that arise in the assessment of teamwork skill development
• Preparation for teamwork
• Focus on process not just product of teamwork
• Clear criteria
• Feedback
• Supporting External Students in Team Selection
• Other issues in the development of the assessment model
• Formulation of the topic
• Timing
• Evaluation of the trial of the assessment model
• Fostering teamwork skills and overcoming pedagogical difficulties
• The Value of Teamwork
• Preparing Students for Teamwork
• Team selection
• Self Selected Teams
• Placed Teams
• Analysis of Assignment Performance
• Key parameters
• Criteria
• Managability
• Suggestions for future implementations of the model
• Timing of the exercise
• Conclusions
• References
• Appendix - Assessment Model
• Notes

Introduction

1. Recent stakeholder pressure to reconsider the fundamental structures and philosophies underpinning legal education provided a unique opportunity to reconsider the design of the curriculum at the Queensland University Technology (QUT) Faculty of Law.[1] Pursuant to a Large Teaching and Learning Grant in 2000-2001, the QUT Faculty of Law redesigned its curriculum to incorporate social, relational and cultural generic capabilities. A series of skills and attributes were identified as being desirable in law graduates and incorporated throughout the courses offered in the Faculty. One of these skills was teamwork.

2. Teamwork boasts a number of significant pedagogical benefits. Where students are working together to achieve understanding, they may go through a process of sharing their own points of view, knowledge and understanding in order to construct a new knowledge through discussion and debate.[2] Where the focus of the teamwork exercise requires understanding rather than retention, students may be more encouraged to adopt a deeper approach to their learning. Further, because learning in a team is collaborative, students have ample scope to take an active role in their learning. The process of meeting and discussing their work fosters and encourages such an active approach. Where students do adopt deeper and more active approaches to learning, the learning process is more likely to be an effective one.[3] Teamwork also provides students with practice and experience in a variety of interpersonal skills preparing them for the reality of the workplace.[4]

3. However, research already done in this area demonstrates that the assessment of teamwork skill development is difficult. Its validity and reliability are often questioned. There are also logistical difficulties involved in fostering teamwork skills in external students.

4. Having made the decision to incorporate skills into the curriculum, one of the more difficult issues became how to engender the selected skills in external students. External students are generally not required to attend on campus, except for the External attendance school which is held over one weekend each semester. At the External attendance school, students are required to attend a three to four hour class in each unit they are undertaking externally. Given this limited attendance on campus, how then, for example, could they be expected to undertake work in teams?

5. University and Faculty policy was in favour of engendering these skills in all our graduates, regardless of whether they had undertaken their studies on an external or internal basis. However, the policy presented obvious equity issues. On the one hand it would clearly be unfair to exclude external students from skill development simply because it presented implementation and assessment difficulties, but on the other would it be fair to ask students living remotely from campus to undertake items of assessment in teams operating predominantly outside the classroom? This dilemma suggested the need to change current teaching and learning approaches.

6. Pursuant to a second Large Teaching and Learning Grant in 2002-2003, the Law Faculty is developing an assessment framework to ensure quality in the assessment of generic capabilities. While one of the core focuses of the second grant is assessment of the development of teamwork skills in students, it has recognised that particular difficulties arise in fostering teamwork skills in external students generally. It was acknowledged that a threshold issue was whether these equity issues could be overcome. It was envisaged that this may be possible if external students undertaking teamwork activities were provided with appropriate scaffolding to encourage and foster teamwork skill development. The assessment model developed pursuant to the second grant for the assessment of teamwork skills focuses on engendering teamwork skills in external students. The model has been trialled in a third year law unit with a substantial cohort of external students.

7. This paper will explore how an assessment model has been devised to meet the traditional difficulties that have arisen in fostering teamwork skills in external students through an analysis of the model currently being trialled. The paper will consider how external students, provided with the appropriate scaffolding, can learn teamwork skills from one another while working in virtual teams on a compulsory assignment through an exploration of student reflection upon their learning experiences while working in teams in the trial of the model. This paper explores and presents the results of that trial.

8. In Section 1 the process underlying the development of an assessment model designed to address many of the problems traditionally arising in fostering teamwork skills in external students and assessing their skill development will be explained, and the issues affecting the development of the model will be critiqued. In Section 2 the results of the trial of the model will be outlined and the effectiveness of the model will be critically analysed. The paper will conclude in Section 3 with suggestions for how the model could be improved upon in the future.

   The development of an assessment model designed to foster teamwork skills in external students

   Development of the Assessment Model

9. The unit chosen for the development and trial of the assessment model was a compulsory unit, LWB334 Corporate Law, which is typically undertaken by third or fourth year students. This unit was chosen as it had a substantial external cohort comprising 95 out of the total unit enrolment of 295 students. It has a teaching team of two full-time members of staff, one part-time member of staff and 4 casual academics. The author is the unit co-ordinator of this unit.

10. The primary objectives of the model were to:
    * Provide appropriate scaffolding to facilitate and foster teamwork skill development in all students, but particularly in external students
    * Address as far as possible the pedagogical difficulties involved in the assessment of teamwork raised in a literature review and
    * Meet criteria specifically developed by the Faculty for evaluation of assessment. The criteria require that assessment be valid, reliable, manageable, authentic and help students to develop in the area being assessed.[5]

11. The assessment model developed for trial is set out in the Appendix. In brief, it was developed following a review of current assessment methods, a literature review and consultation with a number of academics from various faculties at QUT.[6] This process of development is explored below.

    Review of current and alternative approaches

12. The project team reviewed current assessment practices in the Faculty of Law in consultation with academic staff. It found that although teamwork was being explicitly developed in a number of units, only in very few units was an assessment of teamwork conducted.[7] It also revealed that external students were rarely, if ever required to work in teams outside the classroom.

13. The search was then expanded to other faculties at QUT and other universities. The project team considered information publicly available on current practices, experiences and models used in the development and assessment of teamwork skills. This was predominantly done through the review of journal articles where academics had reported on their experiences with the development and assessment of teamwork skills and offered their own critical evaluation of the assessment process.[8] Wider research considered developments in educational theory about the assessment of teamwork skills.[9]

14. This review revealed a number of difficulties with teamwork skill development and its assessment. The project team considered the impact of recent technological developments and how that technology may be implemented in fostering teamwork skill development in external students. Enabling technology and the traditional difficulties arising in teamwork were analysed with a view to devising an assessment model that would seek to foster teamwork skills among internal and external students and also validly and reliably assess teamwork skill development.

    The Impact of Technology on Distance learning

15. Distance education has been defined by UNESCO as the 'variety of educational programs and activities where the learner and teacher are physically separate but ... efforts are made ... to overcome this separation using a variety of media.'[10] Laurillard has noted that communicative media such as email, telephone and video conferencing were originally used in education to communicate with distance learners. However, Laurillard has noted that over the past few years, two significant developments have changed the significance of these media in higher eduction - 'the increase in life long learning and the Web'.[11] Further Laurillard notes that the profile of the undergraduate population has changed, as universities find that the majority of their undergraduate students are part-time, mature students who often already hold another undergraduate qualification.[12] As the profile of our student cohort changes, so too do their needs. This requires a rethinking of the implementation and enhancement of traditionally effective teaching and learning strategies.[13]

16. This is a highly discerning student population, relatively affluent, mobile and hard working. They will demand a lot from the university they return to, and as Palloff and Pratt (1999) point out, the campuses will respond by working hard to create learning communities among these groups. The key environment for this group is the online community. The Web would have been an attractive medium for campus-based students in any case, because unlike the earlier forms of communication over the Internet, it facilitates a much wider range of communicative forms. For distance-learning students, it becomes a lifeline.[14]

17. Distance learning is becoming flexible learning as students use it a device through which to choose the setting in which they will undertake their course.[15] In the QUT Faculty of Law a significant proportion of the undergraduate student body undertake particular units or their entire course in 'external' mode. For example in the unit in which the assessment model was trialled, almost one third of the student cohort was made up of external students.

18. The flexibility and advantages of distance learning are likely to be responsible for its continued popularity.[16] At the QUT Faculty of Law, this popularity of undertaking units externally is increasing, particularly units in the final years of the course, such as the unit in which the assessment model was trialled. One possible explanation for this is that as students near completion of their course, they are more likely to commence professional employment and they enjoy the flexibility of being able to continue their studies without being required to attend on campus. The possibility of undertaking units in external mode facilitates flexible learning by allowing the course to meet the needs of the learner by adapting to their needs as they change throughout their course.[17] In this respect, distance learning is available to students who might not otherwise have been interested or available to enrol in face to face mode.[18]

19. Distance learning is also in favour with academic institutions facing the modern pressures of limited funding, resources and staffing. Allowing students to undertake their studies in external mode can prove 'a cost-effective method of delivering higher education'.[19] This is particularly relevant for space-constrained campuses such as QUT.

20. As a medium which supports discussion, Laurillard heralds the Internet as addressing learning acitivities which enable interaction, reflection, feedback and goal setting. This is based of course on an assumption that 'students can learn through discussion and collaboration, even at a distance and asynchronously'.[20]

21. Collis describes the Internet as enabling 'a new educational paradigm' of 'interconnectiveness'.[21] Enabling interconnectiveness and interactivity are the novel technologies offered by the Internet to distance learning.

22. The concept of interactivity also extends beyond the realms of interactive properties of the instructional technologies to include interpersonal communication between learners and between learners and instructors. There are several important reasons to promote some or all types of interactions in distance learning courses. Some of the reasons suggested by instructors and participants alike include the following:
    • Decreases the sense of isolation of individuals involved in distance courses.
    • Increases the flexibility of individuals to adapt to new conditions.
    • Increases the variety of experiences individual learners are exposed to, such as mulitcultural environments, broader age range of learners or greater overall expertise of all learners combined.[22]

23. These improvements in interactive communicative technologies pave the way for the 'development of new instructional technologies and communication tools that can deliver knowledge without the limitations imposed by traditional learning environments.'[23] In the development of the assessment model it was considered that the Internet would play a key role in fostering teamwork skills in external students. The model acknowledged that external students are not generally required to come onto campus. To address this issue, teams with external student members were encouraged to operate in Internet-facilitated virtual teams with occasional face-to-face meetings where possible.[24]

    Learning in Internet-facilitated Virtual teams

24. The near instantaneous speed and global accessibility of the Internet have considerably affect the manner and alacrity with which people can access and exchange large amounts of information.[25] This enabling technology makes working and studying 'on-line' in cyberspace convenient and practical.[26] Virtual teams are considered to be less disruptive to people's lives than face to face teams, because they obviate the need to travel to meet.[27] The Internet transcends time and distance to enable the team to come to the individual, rather than the individual going to the team.[28]

25. Virtual teams work in a digital reality inciting non-traditional ways of learning collaboratively and organizing teamwork tasks.[29] Organisation of teamwork activities can be enhanced as the electronic organization of data allows communications and documents exchanged to be recorded and stored on a central electronic database accessible only to the members of the team.

26. Virtual teams are a phenomenon of the digital age in which we live and a reality in the workplace. Research into the function of virtual teams and their quantitative and qualitative outcome indicates that they are not intrinsically better or worse than conventional teams operating face-to-face from the same workplace[30] Just as conventional teams face numerous challenges, so to do virtual teams.

27. Although the effective use of electronic communication and collaboration technologies is fundamental to the success of a virtual team, virtual teams entail much more than technology and computers. When virtual teams and their leaders are asked about their successes and failures, they rarely mention technology as a primary reason for either.'[31] The technology simply forms the scaffolding that enables the team to get on with the teamwork.[32]

28. Virtual teams cannot be any more cohesive than a traditional co-located team; therefore it is important to lay a good foundation first.[33]

29. Virtual teams can also be more complex that face to face teams for two reasons:
    1. 'they cross boundaries related to time, distance'[34] (geography); and
    2. 'they communication (share information) and collaborate (work together to produce a product) using technology.'[35]

30. 'As the distance between team members increases, so to do differences in time zones. This makes communicating and collaborating at the same time problematic.'[36] In the trial unit we had students working remotely from Afganistan, Japan, the United Kingdom, Indonesia, Perth, Adelaide, Darwin, Sydney, Melbourne and Queensland.

31. Earlier studies have suggested that virtual teams should over-communicate; virtual team leaders may find that coordinating the team requires more structure and strategic planning than in face-to-face teams; and that the team as a whole should give careful thought to which communication method is most appropriate to achieve the objectives of each meeting.[37]

32. In a learning context where there is no obvious team leader, it was recognised that it was up to the course designer to provide that structure, as part of the scaffolding necessary to ensure an effective team learning experience, especially for external students.[38] The manner in which this scaffolding was erected is outlined in the next section of this paper.

    Analysis of difficulties that arise in the assessment of teamwork skill development

33. 'A good virtual team is, at heart, a good team'.[39] To be a good team it is also necessary to overcome the traditional barriers to teamwork. Following a review of problems usually associated with fostering and assessing teamwork skills, the assessment model specifically sought to overcome a number of difficulties that traditionally arise. These included:
    * lack of instruction on how to work in teams and deal with conflict,
    * focus on the product not the process of teamwork,
    * team selection,
    * lack of, or misleading criteria and
    * a general lack of personalised feedback on skill development.

    The manner in which each of these difficulties has been addressed by the assessment model is outlined below.

    Preparation for teamwork

34. The assessment model sought to prepare students for teamwork in two ways. Firstly materials were developed for inclusion in the comprehensive study guide available to all students in the unit explaining why teamwork skills were being incorporated into their course and how they were to be incorporated and assessed.[40]

35. The materials explained the various stages of team development and offered students guidance on how to operate effectively in a team, how to deal with conflict and further resources appropriate to each stage of team development. In this unit, where teamwork skills are developed at their highest level in the undergraduate program (Level 3),[41] no further instruction on how to operate effectively in a team should have been necessary. Accordingly, the materials were designed to serve primarily as a student-friendly, reference and revision tool, and secondly as a tool to assist students who may not have progressed through the earlier levels of skill development by referring them to resources used in earlier units developing the skill at lower levels.[42]

36. Secondly, external students undertook an on-line tutorial exercise designed to prepare students for teamwork in the unit.[43] Internal students undertook this preparatory tutorial in class. These exercises incorporated reflection on the students' previous teamwork experience, their hopes for their teamwork experience in the unit, the development of ground rules, how to deal with conflict and matters to consider in respect to team formation. In this unit, students were free to team with other students irrespective of whether they were undertaking the unit internally or externally. Students were asked to reflect upon their team formation options and consider what they should look for in potential team mates, based on their own strengths and weaknesses in a team and their goals for this teamwork exercise. Students were encouraged to discuss their reflections with potential team mates.

37. In the tutorial, students considered the variety of options for conducting team meetings in virtual or face to face mode. Both internal and external students were encouraged to think of the benefits of operating as virtual teams. It was considered that given the preponderance of student use of the Internet in their studies, the changing profile of the student cohort and busy lifestyles, it was wise to instruct all students in the theory behind working in virtual teams and the benefits that it may offer them in terms of flexibility.

38. It was anticipated that teams with external student members would meet predominantly in a virtual or on-line fashion and that this would require support. To facilitate this interaction, private discussion forums were set up for each team with an external student member in order to facilitate communication and document exchange.[44] These private discussion forums were accessible via the unit's Internet homepage. The forums were private in that access was restricted only to members of the individual team through password access. These forums allowed students to upload text messages, documents and weblinks. In this manner the forums operated as an effective central database recording team progress, communications and document storage. Team members were able to work as a team on the same document, rather than attempt to collate individual drafts.

39. Further, given that research into the efficient operation of effective virtual teams in a work environment had indicated that virtual teams benefit from at least one face to face meeting, it was decided that every effort should be made to encourage teams with external student members to meet face to face. This was achieved by scheduling 90 minutes at the compulsory External Attendance School for team meetings. Students were advised that thereafter, the unit co-ordinator would be available for consultation should any team wish to discuss any difficulties that the team was facing.

40. Student concern for preparation for teamwork activities was a factor identified by the project team as worthy of further investigation. The outcomes of that further investigation are outlined in Section 2 of this paper.

    Focus on process not just product of teamwork

41. In 2001 the QUT Teaching and Learning Support Services (TALSS) embarked on an investigation of Student Perspectives on the Development of Generic Capabilities at QUT.[45] In this investigation, students complained that when teamwork is assessed, they are typically assessed only on the product of the teamwork rather than the teamwork process. This complaint is closely linked to two issues of pedagogical concern. Firstly, student learning is highly influenced by assessment.[46] If students are instructed on the importance of developing their teamwork skills but the criteria for assessment indicate that it is only the substantive content and not the skill which is to be assessed, students will take this as their cue as to what is really important about the exercise.

42. Secondly, where teamwork is undertaken outside the classroom, staff may have had little or no opportunity to observe the student's development of teamwork skills.[47] In this paper, this difficulty will be referred to as 'observability'. Staff may be assessing the product because it is all that they have to assess. On the other hand, peers have first hand experience with the student in a team. They are clearly in a better position to provide feedback on the teamwork skills of their team members. Consequently, they may also be better placed to assess teamwork skill development.[48]

43. The assessment model incorporated a multi-leveled approach to assessing teamwork skills, designed to facilitate student understanding that the development of teamwork skills is important to them.[49] Staff would assess the product of the teamwork, namely, the assignment. Peers, both internal and external, would provide students with formative feedback on how well they worked within the group dynamic, their strengths, and suggestions for improvement. In addition, peers would give formative and summative feedback on the student's contribution to the team project through self and peer evaluation. This was clearly explained to students in the unit materials, all of which were made available to students prior to the start of the semester.[50] In this manner the assessment model sought to overcome legitimate concerns of inequitable contribution to teamwork.[51]

    Clear criteria

44. In order to enhance the reliability of the model, students were provided with clear criteria, explicitly addressing the assessment of teamwork skill as a component of the exercise.[52] The careful design of such criteria is imperative as student approaches to learning are likely to be highly influenced by assessment,[53] and may even 'define what the student regards as important'.[54] Unless it is indicated in clear and transparent criteria that the development and attainment of skills is to be explicitly assessed, students may interpret that the development of the skill is relatively unimportant.[55]

45. The criteria were designed to encourage desirable approaches to learning by avoiding 'a proliferation of detailed and trivial objectives'[56] which may encourage students into more superficial and surface approaches.[57] Students were required to demonstrate understanding, critical analysis and original thought.[58]

46. Students and staff were educated in the meaning and application of the criteria for assessment of skill development. The criteria had been embedded in a self and peer evaluation instrument.[59] Students were provided with a sample of the evaluation instrument which had been completed by a mock team. External students practiced using the criteria at the compulsory External Attendance School in an exercise offering students an opportunity to discuss the meaning of the criteria and how to complete the evaluations.[60]

47. It was envisaged that these initiatives would enhance the quality of the assessment and learning process and enhance student confidence in the development of their teamwork skills.[61] Confidence and understanding of the criteria for assessment may be closely linked to the quality of peer feedback. Sadler suggests 'one of the conditions necessary for the intelligent use of feedback is that learners know not only their own levels of performance but also the level or standard aspired to or expected'.[62]

    Feedback

48. The model specifically addressed the need for feedback and student reflection on their teamwork skill development. Peers would provide formative feedback on how well the student worked within the group dynamic, their strengths, and suggestions for improvement. In addition, formative and summative feedback on the student's contribution to the team project would be given through self and peer evaluation. Each student was required to complete a reflection on teamwork. This reflection was required to be submitted with the teamwork skills component of the exercise, but was not the subject of assessment. The assessment of student reflection is generally considered to be inadvisable. Given students' capacity to be profoundly affected by assessment, it is likely that students would not undertake a genuine reflection but rather craft their reflections to represent the student's impression of what the academic wants them to be.[63]

49. The reflection sheets used in the assessment model drew from what the project team considered to be the best elements of the reflection sheets already being used in the Law Faculty.[64]

50. Students were essentially asked to reflect upon their strengths in teamwork, what they found most enjoyable or profitable about working with others, what challenged them in teamwork and what they would like to be able to do more effectively. Students were asked to reflect on their feelings about teamwork before and after completing the teamwork components of the unit, the nature of their most effective meetings and the effectiveness of their conflict resolution strategies.

    Supporting External Students in Team Selection

51. In the development of the assessment model, three alternative modes of team formation were considered:
    • Random placement into teams.
    • Placement into teams according to ability or motivation.
    • Self-selection

52. Earlier studies indicate that self-selection tends to lead to partial streaming where better students select one another and friends work together.[65] Lejk, Wyvill and Farrow have observed that where friends work together they may not form rigorous or disciplined groups.[66]

53. Further, external students may find the process of self-selecting teams unduly stressful. It emerged from the TALSS investigation[67] that students were generally concerned about undertaking group work within a competitive learning environment which fostered a sense of isolation, and which had the potential to undermine effective teamwork.[68] Isolation felt by external students may be compounded by their remote location and social isolation from other students.

54. However, there are difficulties with both the alternative methods of team selection. Earlier studies have indicated that placing students in teams either randomly or according to ability can adversely impact on student performance and motivation.[69]

    '[H]igh ability students were motivated and had a positive learning experience when working with other high ability students and there may have been a demotivating influence when they worked with students at the bottom end of the range. On the other hand students in the lowest range were perhaps motivated when they worked in mixed ability groups and demotivated when they worked with others of similar ability.'[70]

    These difficulties with team formation were a factor identified by the project group as worthy of further investigation. The outcomes of that further investigation are outlined in Section 2.

55. In the assessment model students were required to form teams of 3-4 students. Students were given two options for team formation. They could either choose their own team-mates by a given date (the self-selection option), or be placed in a team by staff.

56. Although the unit was developing teamwork skill at the highest level of undergraduate skill development (Level 3), it was anticipated that as skills had only been recently inculcated into the curriculum, a significant number of students may not have undertaken earlier units at a time when they had incorporated teamwork skill development. This factor, coupled with the studies on the impact of team formation on motivation, was highly influential in the decision to include a self selection option.[71] To address the issues identified by Lejk et al in earlier studies, the materials developed for the unit warned students against teaming with friends and offered advice on what to look for when selecting team-mates.[72]

57. To facilitate opportunities for self-selection among external students, an on-line discussion forum was set up on the unit's on-line teaching website. While this forum was available for use by all students, it was a particularly popular and effective tool with external students. External students using this forum were generally explicit in what they were looking for in a team-mate in terms of ability, motivation, availability and location.

58. The placement option was included to assist external students with concerns of isolation, stress in team formation and because, at this level of skill development it is appropriate that team formation as far as possible replicate team formation in the workplace.

59. Although this option was available to all students, it was anticipated that external students would be most likely to require placement in teams. External students were also offered additional assistance in team formation: if they did not choose to self-select a team by a given date, then every effort was made to place them with students living in their local area to facilitate ocassional face-to-face meetings.[73]

    Other issues in the development of the assessment model

    Formulation of the topic

60. The assignment topic was carefully designed to ensure that it was not clear on its face that it could be divided into components parts.[74] Rather, after preliminary research students were expected to discover that the focus of the assignment required them to read and understand three complex cases revealing significant deficiencies in the legislative scheme underlying the area of law being studied in this unit. Students were then required to critically analyse attempts to overcome the deficiencies. It was considered likely that students would assign a team member to each case and discover that they would be inclined to need to collaboratively discuss each of the three cases before embarking on the critical analysis. It appears from information provided by students in their assignments that this is what happened.

    Timing

61. The topic for this item of assessment and all materials were available to students before the start of the semester. Team formation was finalized at the end of week 3 and the assessment was not due until Week 9. These time limits were set to ensure that students sorted out their teams as early as possible, settled into the team and began their teamwork as early in the semester as possible. It also allowed sufficient time for virtual teams to overcome possible technology issues, and delays caused as a result of having team members in significantly different time zones.

    Evaluation of the trial of the assessment model

62. The trial appears to have been remarkably successful in achieving its objectives of fostering of teamwork skills in external students, overcoming the traditional pedagogical difficulties that arise in the fostering and assessment of teamwork skill development. This part of the paper will explore the extent to which the trial of the model has achieved its objectives.

63. In order to evaluate the trial of the model, the project team sought feedback from a number of participants in the trial, namely:
    • Staff responsible for the administration of the unit trialling the model.
    • Staff at the coal-face of preparing students for teamwork and marking the assignments.
    • External and internal students who completed the trial of the model.

64. Student perspectives were obtained in two ways. Firstly, de-identified data from the students' reflections on teamwork has been quantitatively analysed. Secondly, all students undertaking the unit were asked to complete a confidential survey seeking their feedback on the assessment of skills in the unit undertaking the trial. [75]

65. The aim was to survey all students internal and external. Surveys were completed by 183 of the 295 students enrolled in the unit. External students were surveyed at the compulsory External Attendance School. Internal students were surveyed in their usual tutorial groups. The surveys were completed by 52 external students and 131 internal students. Although the survey was also made available on-line, no other surveys were completed. The sample is considered to be representative.[76]

66. Responses to the survey and the quantitative analysis of student reflections were dissected to ascertain whether there had been a significant difference in response between internal and external cohorts. On the whole the responses from both cohorts were reasonably consistent; however there were some questions where the strength of the response was greater in one group. This analysis will particularly focus on external student feedback.

    Fostering teamwork skills and overcoming pedagogical difficulties

67. This part of the paper will explore the extent to which the assessment model has successfully fostered teamwork skills in external students and overcome the traditional pedagogical difficulties that arise in the assessment of teamwork skills. In examining these issues, results have been presented from the student survey as well as an analysis of assignment performance in various categories of interest.

    The Value of Teamwork

68. While the value of teamwork will not be an area of focus in the testing of the Draft Assessment Framework, it was considered that it would be a useful context in which to place the evaluation of the model. Given that external students had rarely been required to undertake teamwork acitivities outside the classroom in their earlier studies in the Law Faculty, it was expected that there would be some resistance and reluctance to embrace teamwork as an effective learning tool. To this end, responses were solicited from students to confirm student attitudes to teamwork and the importance of embedding and developing their teamwork skills. Appearing below are a selection of comments from external students working in virtual teams as to their experiences with the teamwork exercises in the assessment model.

    'Due to the nature of my work, I am very rarely able to participate in team or group acitivities in my course. This is the one thing I miss about internal studies many years ago. This is obviously contingent upon establishing a good group. It was therefore a pleasure to work with a group of students. It also helped that they were pro-active and flexible. Teamwork is a very large part of my professional career (armed forces) and I therefore am no stranger to the notion. However this was a pleasant change. It has changed my mind about the idea of 'blind dating' and establishing teams with students I do not know.'

    'I have enjoyed the exposure to other perspectives on assignment planning and research. My ideas have definitely broadened. It was also beneficial to always have the group available and receptive to discuss concepts and problems.'

    'I have found that working with others provides both a more comprehensively researched and polished final product.'

    'I found that working with this team was very different from group work in the past as we all contributed equally and had similar goals and attitudes towards studying. Often I prefer to work by myself, rather than with unproductive team members not pulling their weight. However, in this group, I thought it was great how we could use three heads instead of one.'

69. Students were asked in the confidential survey: At the beginning of this semester, how did you feel about working in teams, and how do you feel now? The quantitative analysis of student reflections on teamwork revealed that although most students felt negative about teamwork at the beginning of the semester, their feelings have now changed. 86.7% of external students indicated that they felt negative about teamwork at the start of the semester. Of those students, only 13% indicated that they had emerged from their teamwork experience in this unit still feeling negative. Appearing below are a selection of responses from external students working in virtual teams responding to survey question: How did you feel about working in teams, and how do you feel now?

    'I was dreading it. I did not want to be part of a team at all. I thought it was a big waste of time. ...NOW... I can really see the benefits, especially if you are working with people you really trust. I feel my team members were so supportive that they brought out the best in me.'

    'At first I thought this assignment would be a waste of my time. I was very cynical about the applicability of this sort of team exercise for the real world. ...NOW... I feel that I have learned a lot about the dynamics of teams and how best to manage projects. We had the opportunity to see different leadership styles and profit from the strengths that each team member brought to the group.'

    'I have always been apprehensive about working in teams. As the mother of two children, working full time and studying a full time load, I am very used to working at my own pace. ...NOW... I realise that a far better assignment can be produced. You are able to get the best of everyone's ideas and input and weed out the weaker arguments. Through that you gain a more thorough understanding of what you are working on as you have others there to bounce ideas and questions off. I feel more confident working in a virtual team and now realise the extent to which technology can make teamwork flexible.'

    'I have worked in a team environment on a professional basis and have always found it difficult. ...NOW... This experience has been refreshing. I would go so far as to suggest that the three of us generally complemented each other. I would be happy to work in a team again'. [This comment was from a student who, prior to the commencement of the exercise, had lodged an administrative complaint that it was inequitable to insist that external students work in teams].

70. These comments are typical of student feedback and are reinforced by responses from the survey which revealed that students appeared to have a positive attitude to the inclusion of teamwork skill development in the unit. Almost 85% of external students surveyed indicated that they could see the point of developing teamwork skills. 76% agreed that developing teamwork skills was of benefit to them. These findings are demonstrated in the Tables 1 and 2.

    Table 1

    I can see the point of developing teamwork skills

    	All Students		External		Internal
    	#	%	#	%	#	%
    Strongly Agree/Agree	159	86.9%	44	84.6%	115	87.8%
    Neutral	14	7.7%	7	13.5%	7	5.3%
    Strongly Disagree/Disagree	10	5.5%	1	1.9%	9	6.9%
    No answer	0	0.0%	0	0.0%	0	0.0%
    Total respondents	183	100.0%	52	100.0%	131	100.0%

    Table 2

    Developing my teamwork skills is of benefit to me

    	All Students		External Students		Internal Students
    	#	%	#	%	#	%
    Strongly Agree/Agree	133	72.7%	40	76.0%	93	71.0%
    Neutral	28	15.3%	6	12.0%	22	16.8%
    Strongly Disagree/Disagree	20	10.9%	6	12.0%	14	10.7%
    No answer	2	1.1%	0	0.0%	2	1.5%
    Total respondents	183	100.0%	52	100.0%	131	100.0%

     

71. Over 70% of external students indicated that the teamwork exercises in this unit had improved their teamwork skills. This trend was not as strong in internal students. One explanation for this might be that external students in the Law Faculty are typically not required to undertake teamwork activities. A majority of all students indicated that the teamwork exercises in this unit had helped their understanding of the unit content more than working alone.[77] These findings are indicated in Tables 3 and 4.

    Table 3

    My ability to work in a team has improved as a result of studying in this unit

     

    	All Students		External		Internal
    	#	%	#	%	#	%
    Strongly Agree/Agree	109	59.6%	37	71.2%	72	55.0%
    Neutral	59	32.2%	10	19.2%	49	37.4%
    Strongly Disagree/Disagree	15	8.2%	5	9.6%	10	7.6%
    No answer	0	0.0%	0	0.0%	0	0.0%
    Total	183	100.0%	52	100.0%	131	100.0%

     

    Table 4

    Teamwork has helped my understanding more than working alone

     

    	All Students		External		Internal
    	#	%	#	%	#	%
    Strongly Agree/Agree	113	61.7%	33	63.5%	80	61.1%
    Neutral	46	25.1%	12	23.1%	34	26.0%
    Strongly Disagree/Disagree	24	13.1%	7	13.5%	17	13.0%
    No answer	0	0.0%	0	0.0%	0	0.0%
    Total	183	100.0%	52	100.0%	131	100.0%

    
    

72. These findings tend to lend support to the underlying premise for embedding teamwork as a skill and a valuable teaching and learning tool within the curriculum for both internal and external students.

    Preparing Students for Teamwork

73. The survey revealed that students considered that they had sufficient preparation for teamwork. 80% of external students surveyed indicated that they did not require further resources. External students gave particularly strong responses to questions pertaining to resources. Over 88% of external students indicated that the resources were good for revising teamwork and almost 85% indicated that they were adequate for reference. This trend was not as strong in internal students. These findings are demonstrated below.

    Table 5

    Do you require further resources?

    	All Students		External		Internal
    	#	%	#	%	#	%
    Yes	6	3.3%	1	1.9%	5	3.8%
    No	161	88.0%	42	80.8%	119	90.8%
    No answer	16	8.7%	9	17.3%	7	5.3%
    Total	183	100.0%	52	100.0%	131	100.0%

    Table 6

    Resources were good for revising teamwork

    	All Students		External		Internal
    	#	%	#	%	#	%
    Strongly Agree/Agree	124	67.8%	46	88.5%	78	59.5%
    Neutral	42	23.0%	4	7.7%	38	29.0%
    Strongly Disagree/Disagree	17	9.3%	2	3.8%	15	11.5%
    No answer	0	0.0%	0	0.0%	0	0.0%
    Total	183	100.0%	52	100.0%	131	100.0%

    Table 7

    Resources were adequate for reference

    	All Students		External		Internal
    	#	%	#	%	#	%
    Strongly Agree/Agree	132	72.1%	44	84.6%	81	61.8%
    Neutral	47	25.7%	7	13.5%	39	29.8%
    Strongly Disagree/Disagree	3	1.6%	1	1.9%	11	8.4%
    No answer	0	0.0%	0	0.0%	0	0.0%
    Total	183	100.0%	52	100.0%	131	100.0%

    
    

74. These results are particularly encouraging given the importance which was placed on providing external students in particular with the 'scaffolding' required to facilitate learning in virtual teams. This is further reinforced by the fact that 80.4% of all students surveyed (both internal and external) indicated that their most effective meeting was a held in 'virtual' mode. In their reflections, as to what they would have liked to change about their team, even students who indicated that their most effective meeting was a virtual one, indicated a preference for face to face meetings. Perhaps these face to face meetings were preferable for the increased synergy that is associated with meeting face to face or because they were more socially interactive and therefore more enjoyable. The most frequent student responses are summarised in the table below.

    Table 8

    Top 5 Responses:  If you could change one thing about this team what would it be?

    

     

     

     

     

     

     
    

    Team selection

75. An analysis of team formation revealed that external students had been more likely to require placement into teams. This issue had been ear-marked for further investigation as to whether team selection had an impact upon student performance.

     

    Table 9

    Team Selection Patterns

     

    	Self Selected		Placed
    Cohort	%	#	%	#	Total
    External Students	53.7%	51	46.3%	44	95
    Internal Students	77.5%	155	22.5%	45	200
    All Students	69.8%	206	30.2%	89	295

     

76. To further explore this issue the survey solicited explicit responses in relation to self-selected and placed teams.

    Self Selected Teams

77. Students indicating that they had self-selected their own team also made the following indications: internal students were more likely to indicate that selecting their own team had minimised their stress in team formation, than external students. Only 50% of external students who had selected their own team felt that it had been the best team formation option for them. This trend was not reflected among internal students, who gave particularly strong responses in support of their choice to select their own team. These findings are summarised in Tables 10 and 11.

     

    Table 10

    Preselecting my own team minimised my stress in team formation

    	All Students		External		Internal
    	#	%	#	%	#	%
    Strongly Agree/Agree	103	73.0%	18	60.0%	85	76.6%
    Neutral	24	17.0%	7	23.3%	17	15.3%
    Strongly Disagree/Disagree	6	4.3%	1	3.3%	5	4.5%
    No answer	8	5.7%	4	13.3%	4	3.6%
    Total	141	100.0%	30	100.0%	111	100.0%

    Table 11

    Preselection was the best option for me

    	All Students		External		Internal
    	#	%	#	%	#	%
    Strongly Agree/Agree	109	77.3%	16	53.3%	93	83.8%
    Neutral	17	12.1%	8	26.7%	9	8.1%
    Strongly Disagree/Disagree	7	5.0%	2	6.7%	5	4.5%
    No answer	8	5.7%	4	13.3%	4	3.6%
    Total	141	100.0%	30	100.0%	111	100.0%

     

78. The capacity for students to select their own team appears to have been a significant influence in their attitude to teamwork. The attitudes of students to teamwork at the start of the semester and at the conclusion of the teamwork component of the unit indicated that students had overwhelmingly left with a more positive attitude to teamwork.[78]

    Placed Teams

79. One of the reasons placement into teams was included as an option in team formation was based on the assumption that external students may face undue stress in forming their own teams, given that they did not have the same opportunity as internals students to pre-select themselves based on friendships or prior associations. Further, the capacity and inclination of external students to form their own teams may be more affected by factors such as isolation and distance. The scores for 'placement was the best option for me' and 'placement into a team minimised my stress in team formation' were dissected for responses received from internal and external students. The results of this further analysis tend to lend weight to that proposition. This is illustrated below.

    Table 12

    Placement was the best option for me

    	Internal		External
    	#	%	#	%
    Strongly Agree/Agree	9	45.0%	16	72.7%
    Neutral	5	25.0%	6	27.3%
    Strongly Disagree/Disagree	6	30.0%	0	0.0%
    No answer	0	0.0%	0	0.0%
    Total	20	100.0%	22	100.0%

    Table 13

    Placement minimised my stress in team formation

    	Internal		External
    	#	%	#	%
    Strongly Agree/Agree	10	50.0%	15	68.2%
    Neutral	4	20.0%	3	13.6%
    Strongly Disagree/Disagree	5	25.0%	4	18.2%
    No answer	1	5.0%	0	0.0%
    Total	20	100.0%	22	100.0%

80. These results lend support to the proposition that placement into teams is necessary to assist external students in overcoming the impact of isolation on their teamwork experience. To this end, even if external students are required to operate in virtual teams, it is still preferable that they be placed with students in their local area, where possible. Each of the results of the student survey and the quantitative analysis of student reflections, have been analysed to determine differences in external students and internal students. On the whole the most significant differences are that internal students were more able to facilitate self-selection than their external counterparts and that external students indicated to a much greater extent than internal students, that their teamwork skills had improved as a result of studying this unit.[79] Otherwise, feedback on the learning experiences of external and internal students undertaking this teamwork exercise has been reasonably consistent.

    Analysis of Assignment Performance

    Key parameters

81. In addition to the survey, statistical information on the significance of team selection issues was obtained by recording the results achieved by teams with various characteristics. The analysis is a high level summary analysis focusing on a comparison of mean scores of various categories of interest within the overall sample. Key parameters which were examined were
    • External Students vs Internal Students
    • Placed vs Self-selected Groups
    • Team Size (2, 3 or 4)[80]
    • The stated existence of conflict within the groups

82. These parameters were compared both in isolation and in combination to determine which factors or combination of factors had the greatest impact on overall performance. This investigation revealed that:
    • self-selected teams performed significantly better on average than placed teams. This would tend to lend support to the link between self selection and streaming referred to in Section 1.1 of this paper.
    • Teams of internal students performed marginally better on average than external students.
    • Teams of three performed better than team of four (however the difference was not significant). Teams of two performed significantly worse than either teams of three and four. A most likely explanation of this is that these teams may have been adversely affected by the withdrawal of a team-member after the teamwork exercise had commenced.
    • The effect of conflict did not appear to have any bearing on overall performance.

83. These results tend to support the conclusion that the assessment model was successful in its efforts to foster teamwork skills in external students, relative to experiences in previous studies in terms of narrowing the gap between internal and external student performance. In previous experiences of external teamwork performance in other units, external students had performed worse than internal students.

84. For example, in a recent experience in a second year undergraduate core unit in the Law Faculty, internal and external students were both asked to undertake the same teamwork activity in class. External students undertook the activity at the External Attendance School. Despite the fact that the exercise was marked by the same markers, external students, on average received marks 10% lower than that of their external counterparts. In subsequent offerings of the unit, this has not occurred as efforts have been made to ensure that external students are more prepared to undertake the teamwork exercise. These efforts have included having the students complete a formal assignment on the material covered in the exercise prior to completing the teamwork exercise, increasing the time in which external students undertake the exercise and providing formal instruction on how to work in a team. The results of the student survey and the quantitative analysis of student reflections on the assessment of teamwork in the trial tend to indicate that this has been a positive learning experience in teamwork skill development for both internal and external students.

    Criteria

85. Staff responsible for the marking the assessment also provided feedback on the criteria developed for the assessment model. Moderation indicated that no adjustment would be necessary as the average mark and standard deviations of each of the markers were remarkably similar. Although the average mark was quite high, staff indicated that overall the quality of the assignments was particularly good.[81] This was not particularly remarkable as earlier studies have indicated that teamwork 'often results in very good work being produced for which grades are both higher than usual and in a narrower range'.[82] Indeed that was the case in the trial of the model.[83]

86. Staff indicated that the criteria for marking the essay and research methodology were easy to work with from the marker's perspective. It appears that staff have also interpreted the criteria consistently, making moderation unnecessary. In addition, 80% of students surveyed indicated that the criteria for assessment of teamwork were clear. These findings are set out in Table 14.

    Table 14

    The Teamwork Assessment Criteria are clear

    		All Students		External		Internal
    		#	%	#	%	#	%
    	Strongly Agree/Agree	147	80.3%	44	84.6%	103	78.6%
    	Neutral	27	14.8%	4	7.7%	23	17.6%
    	Strongly Disagree/Disagree	6	3.3%	1	1.9%	5	3.8%
    	No answer	3	1.6%	3	5.8%	0	0.0%
    	Total	183	100.0%	52	100.0%	131	100.0%

     

87. This result for external students is significant considering the importance which was placed on clear assessment criteria as part of the 'scaffolding' to support learning by external students.

    Manageability

88. A key concern of the project team with the assessment model, was that it would not satisfy the criteria of manageability from the perspect of staff required to administer the model. However, the model was surprising managable.

89. The more time consuming administrative matters were placing students into teams and adjusting to the fluctuations in unit enrolment in the early weeks of the semester. The on-line discussion forum set up to assist students to find team-mates was a popular and successful tool. It is considered to have reduced the administrative demands placed upon the unit co-ordinator in team formation, especially for external students.

90. An administrative issue with the placed teams, was ensuring that students were able to make contact with their team mates although the methodology underlying team placement resolved these difficulties to some extent. External students were encouraged to contact one another via the discussion forum used for self-selection. Internal students placed together in teams were introduced to one another by their tutors.

91. The discussion forums have proven to be a particularly useful tool in facilitating a number of aspects of the teamwork experience for external students and it is recommended that it be consistently utilised in external student learning at QUT.

    Suggestions for future implementations of the model

    Timing of the exercise

92. Clearly the earlier that the teams are formed and the later the assignment is due, the more time the team will have to become established. However, if the teams are formed too early, there may be greater administrative effort involved in dealing with movements in student enrolments. Also the longer teams have to form, the more opportunity students have to self-select their own teams. The self-selected teams caused absolutely no administrative difficulty. However, the later the assessment is due, the more pressure the markers will be under to return the assignments to students prior to the end of the semester in order to provide timely feedback.

93. In future offerings of this unit, the model with be altered as follows:-
    1. Students will have an additional week to form teams.
    2. The assessment will remain due in week 9.
    3. In order to encourage students to establish contact earlier and ensure that they are holding meetings, (and have practised documenting meetings), the research methodology will be due in week 6, and will take the form of the minutes of the meeting where the students planned their approach to the research assignment. The minutes will be tabled in a tutorial for peer review.[84] This will ensure that students are practising documenting their meetings, that they have established contact, that they have planned their research.
    4. For external students, the practice of having face to face meetings where possible will be further encouraged, with perhaps more time allowed at the External Attendance School for an open discussion of student experiences with virtual teamwork.

    Conclusions

94. This paper has explored the difficulties that have traditionally arisen in fostering teamwork skills in external students. It has outlined the approach taken to foster teamwork skills in external students through an analysis of an assessment model that has been trialled in a third year undergraduate law unit. External students may particularly benefit from teamwork activities, given their propensity to physical and social isolation from the rest of the student cohort. External students clearly need to be provided with the appropriate scaffolding to support their learning. This may include materials on how to work in a team generally and in virtual teams specifically, on-line teaching tools and discussion forums to facilitate communication and document exchange, opportunities to meet face to face, and flexible options in team formation. It is likely that the additional scaffolding provided has resulted in higher standards for external students than would have been achieved otherwise. Students have also indicated that, in their opinion, the exercise has improved their teamwork skills and has helped them to understand the unit material better than they would on their own. The results of the student survey and the quantitative analysis of student reflections on the assessment of teamwork in the trial tend to indicate that this has been a positive learning experience in teamwork skill development for both internal and external students.

References

Australian Law Reform Commission (1999), Managing Justice - A Review of the Federal Civil Justice System, ALRC, Report No 89, esp Chapter 2 "Education, Training and Accountability" citing the American Bar Association (1992), Legal Education and Professional Development -An Educational Continuum, ABA Chicago ("MacCrate Report").

Belanger, F., and Jordan, D.H., Evaluation and Implementation of Distance Learning:  Technologies, Tools and Techniques (Idea Group Publishing, Hershey, 2000).

Boud, D., Cohen, R., and Sampson, J., ‘Peer Learning and Assessment’ (1999) 24 (4) Assessment and Evaluation in Higher Education 413.

Butcher, AC., Stefani, LAJ., Tario, VN., ‘Analysis of peer, self and staff assessment in group project work’ (1995) Assessment in Education 165.

Cheng, W. & Warren, M., ‘Making a Difference:   Using Peers to Assess Individual Students’ Contributions to a Group Project’, (2000)  5 (2) Teaching in Higher Education 243.

Collis, B., Tele-learningin a Ditigal World:  The Future of Distance Learning (International Thomson Computer Press, London, 1996).

Collis, B., and Moonen, J., Flexible Learning in a Digital World:  Experiences and Expectations (Kogan Page, London, 2001), p 9

Conway, R., Kember, D., Sivan, A., Wu, M., ‘Peer Assessment of an Individual’s Contributions to a Group Project’, (1993) 18 (1) Assessment and Evaluation in Higher Education 45. 

Duarte, D. and Snyder,N., Mastering Virtual Teams:  Strategies, Tools and Techniques that Succeed, (Jossey-Bass Publishers, San Francisco 2001).

Falchikov, N. & Magin, D., ‘Gender Bias in Peer Marking’, (1997) 22 (4) Assessment and Evaluation in Higher Education 385.  

Freeman, M., ‘Peer Assessment by Groups of Group Work’, (1995) 20(3) Assessment and Evaluation in Higher Education 289.

Garvin, J.W. et al, ‘Group Projects for First-year University Students:  An evaluation’, (1995) 20 Assessment and Evaluation in Higher Education 273.

Gatfield, T., ‘Examining Student Satisfaction with Group Projects and Peer Assessment’ (1999) 24 (4) Assessment and Evaluation in Higher Education 365

Gibbs, G., Assessing More Students, (Polytechnics and Colleges Funding Council, London, 1992)

Gibbs, G., Learning in Teams:  A Tutor Guide, (Oxford Centre for Staff Development, Oxford, 1995).

Goldfinch, J. and Raeside, R., ‘Development of a peer assessment technique for obtaining individual marks on a group project’ (1990) 15 (3) Assessment and Evaluation in Higher Education 210

Gueldenzoph, L., and May, G., ‘Collaborative Peer Evaluation:  Best Practices for group member assessments’ (2002) Mar Business Communication Quarterly 9.

Hanrahan, S. & Isaacs, G., ‘Assessing Self and Peer-assessment:  the students’ views’ (2001) 20 (1)  Higher Education Research and Development 53.

Hart, G. et al, Student Perspectives on the Development of Generic Capabilities at QUT:  Draft Report, presented to the QUT Teaching and Learning Committee, 30 October 2001. 

Henry, J. and Hartzler, M., Tools for Virtual Teams (ASQ Quality Press, Milwaukee, 1998). 

Hoefling, T., Working Virtually:  Managing People for Successful Virtual Teams and Organisations, (Stylus Publishing, Virginia, 2001).

James, P., ‘A Blueprint for Skills Assessment in Higher Education’, (2000) 25 (4) Assessment and Evaluation in Higher Education 353.

Kift, S., ‘First Year Curriculum Review for Generic Competency:  A Law Degree Case Study’, at the Fourteenth International First Year in Education Conference, University of Canterbury, Christchurch, July 2002 [http://www.qut.edu.au/daa/asdu/fye/abstracts02/KiftAbstract.htm  (Accessed 1 November 2002)].

Laurillard, D., Rethinking University Teaching:  A Conversational Framework for the effective use of learning technologies ( 2^nd ed, Routledge, New York, 2002).

Le Brun, M. and Johnstone, R., The Quiet (R)evolution:  Improving student learning in law, (The Law Book Company Limited, Sydney, 1994),

Lejk, M., Wyvill, M., and Farrow, S., ‘A survey of methods of deriving individual grades from group assessments’ (1996) 21 Assessment and Evaluation in Higher Education 267. 

Lejk, M., Wyvill, M. & Farrow, S., ‘Group Assessment in Systems Analysis and Design:  a comparison of the peformance of streamed and mixed ability groups’, (1999) 24 (1) Assessment and Evaluation in Higher Education 5.

Lipnack, J. and Stamps, J., Virtual Teams:  People Working Across Boundaries with Technology (John Wiley & Sons Inc., New York, 2000).

Magin, D., ‘A Novel Technique for Comparing the reliability of Multiple peer Assessment with Single Teacher Assessments of Group Process work’, (2001) 26(2) Assessment and Evaluation in Higher Education 139. 

O’Donovan, B., Price, M. & Rust, C., ‘The student experience of criterion-referenced assessment’,(2000)38(1) Innovations in Education and Teaching International 74.

Orsmond, P., Merry, S. & Reiling, K., ‘The use of student derived marking criteria in peer and self-assessment’, (2000) 25(1)Assessment and Evaluation in Higher Education 23.  

Queensland University of Technology, Faculty of Law (2001) Overview of  Attributes and Skills,  [https://olt.qut.edu.au/int/lawgrant (Accessed 1 November 2002)].

Queensland University of Technology, Faculty of Law (2001) Table of Core Skills,  [https://olt.qut.edu.au/int/lawgrant (Accessed 1 November 2002)].

Queensland University of Technology, Faculty of Law (2002), ‘Teaching and Learning Development Large Grants Scheme:  Interim Report’, [https://olt.qut.edu.au/law/ASSESSMENT/sec/index.cfm?fa=displayPage&rNum=576131 (Accessed 12 November 2002)].

Radin, M.J., Rothchild, J.A., and Silverman, G.M., Internet Commerce:  The Emerging Legal Framework, (Foundation Press, New York, 2002).

Ramsden, P., Learning to Teach in Higher Education, (Routledge, London, 1992).

Reynolds, A., ‘E-Auctions:  Who will protect the Consumer?’, (2002) 18 Journal of Contract Law 75. 

Stewart, S. and Richardson, B., ‘Reflection and its Place in Curriculum on an Undergraduate Course:  should it be assessed?’ (2000) 25(4) Assessment and Evaluation in Higher Education 369.

UNESCO, Distance Learning Systems and Structures:  Training Manual, Report of a Sub-Regional Training Workshop, Vol. II (UNESCO, Bangkok, 1987).

Van den Brande, L., Flexible and Distance Learning, (John Wiley, Chichester, 1993), p xxi.

Appendix - Assessment Model

ASSESSMENT MODEL TRIALLED IN LWB334 CORPORATE LAW

SEMESTER 2, 2002

 

The detailed study guide available to all students in the unit explains the following:

 

·         The reasons for development of skills in the curriculum,

·         The nature of each explicit skill to be developed in the unit and the stage to which these skills will be developed in the unit.  At this level skills are developed to the final stage of skill development. 

·         That skill development in the unit is part of a wider program of skill development.  In particular, it is explained to students that these skills have been developed in an incremental, horizontal and vertical manner commencing in the first year curriculum and beyond. 

·         The process of skill development.  Each skill is developed through a staged process including instruction on the skill, time and opportunity to practice and reflect, formative feedback and summative assessment. 

·         The platform of achievement expected to be attained by students for each explicit skill on completion of the unit.  This information is extracted from the Table of Core Skills (QUT, 2001). 

The Assessment process in LWB334

While the assessment process is designed to provide a reliable measure of skill development, it is also an effective learning task in its own right embedded within the substantive content.   The written instructions on the assessment task provided to students in their study guide are set out below:

Compulsory Group Assignment

 

The assignment is compulsory and is a measure of your attainment of unit objectives 1-6(c), (d) and (e).

Unit Objectives

 

At the conclusion of this unit, you will be able to demonstrate:

1. knowledge and understanding of the basic legal principles and the policy issues inherent in those principles relevant to registered companies.
2. the ability to analyse and synthesise the legal principles emerging from the relevant statute law and case law with respect to registered companies.
3. knowledge and appreciation of contemporary legal developments and likely future developments through law reform with respect to registered companies.
4. an ability to give legal advice on hypothetical questions relating to registered companies.
5. an ability to critically analyse legal principles with respect to registered companies and to scrutinise the values and ideologies supporting those principles.
6. skills of (a) effective oral communication, (b) problem solving, (c) legal research, (d) teamwork and (e) written communication within the Corporate Law context.. 

 

The assignment topic is set out in the study guide.

 

Working in Teams: The assignment is to be researched and written by a team of three to four students.  Each team is to produce one assignment.  The team approach is designed to assist students in the transition from individual learning at university to performing as a member of a team in the work place.  You should carefully read the material on teamwork available in the study guide, on CMD and on OLT.

 

Team Formation: Team formation is your responsibility.  You should carefully consider the material in the study guide on Forming Teams.  Unless you choose to form your own team in accordance with the Self-selection Option below, the unit coordinator will place you in a team:

 

·         Self-selection Option: You may form your own teams with internal or external students.  If you choose to pre-select your team, you must advise the unit coordinator of the name and student number of each of your team members by (end of week 3 of semester – insert date).  Your team membership will be final once you have advised the unit coordinator of your team details.

 

Or

 

·         Placement in a Team: If you do not notify the unit coordinator of the particulars of your self-selected team by (end of week 3 of semester – insert date), then she will automatically place you in a team.  You will not be placed in a team which has self selected.  Your team membership will be final once you have been advised by the unit coordinator of your team details.

 

Team Acknowledgement: A Team Assignment Acknowledgment Form, signed by each student, must be attached to the assignment.

 

Research Methodology:  As part of the assignment, your team will be required to complete a Research Methodology.  In the Research Methodology you must outline the steps you took in planning, preparing and undertaking the research assignment.  Further details of how to complete the Research Methodology and samples of a completed Research Methodology have been placed on OLT.

 

Teamwork Portfolio: As part of the assignment you are required to outline the steps taken by your team to implement your research methodology.  The teamwork portfolio must include a one page summary of how your team operated supported by select copies of team meeting agendas, minutes, reports and action sheets. Aside from the summary, all of these documents should have been generated at your team meetings.  The page limit will restrict the number that you can select to incorporate in your Teamwork Portfolio.  Select those typical of your meetings and those which substantiate the comments made in your summary of teamwork and your self and peer evaluations.  The following resources are available on OLT to assist you in the development of your teamwork portfolio:

 

·         Further information on how to develop your Teamwork Portfolio can be found under the link to Teamwork Skills Resources

·         Samples of a completed teamwork portfolio

·         Samples of completed minutes, agendas and action sheets

·         Proformas of minutes, agendas and action sheets for you to download and use at your team meetings.

 

Self and Peer Evaluation of your Teamwork: You will be required to submit a self and peer evaluation assessing the performance of each of your team-members and yourself.  This evaluation helps to determine your individual contribution to the team project, including planning and implementation as well as an evaluation of your ability to function effectively in a team.  The self and peer evaluation sheets are available on the OLT site.  You will also find on OLT samples of completed self and peer evaluations.

 

Additional Support for External students: External students will be offered additional support with the compulsory group assignment.  This additional support will be three-fold:

 

·         Team Formation: External students are free to choose either of the team selection options set out above.  However, if you do not choose the Self-selection Option you may also request that the unit coordinator attempt to place you in a team with students living in your area.  Such a request must be received by the unit coordinator no later than (end of week 3 of semester – insert date).

·         Communication and Document Exchange: External students will be offered additional support in order to facilitate communication and document exchange between team members.  A private discussion forum will be set up for teams with external student members.  The forum can only be accessed by members of your team.  You can use the forum to communicate almost instantaneously with one another.  You can also upload documents and links to web pages on your forum. 

·         Opportunity for Virtual Teams to meet face-to-face:  Research indicates that virtual teams benefit from at least one face to face meeting.  Time has been scheduled at the external attendance school for your team to have a face to face meeting.  There are no formal activities scheduled for this meeting time.  You may prefer not to conduct a formal meeting in this time, but rather to use this as an opportunity for your team to meet one another.  The unit co-ordinator will be available for extended consultation at the external school, should you wish to discuss any aspect of the teamwork activity. 

 

 

Calculating your assignment mark: The compulsory group assignment will be worth 20% calculated as follows:

 

·         10%      Substantive Content of Assignment

·           5%      Research Methodology

·           5%      Teamwork Score.

 

The substantive content of your assignment and the Research Methodology will be assessed by a member of the teaching team in accordance with criteria set out in the study guide.  Each team member will receive the same mark for the substantive content of the assignment and for the Research Methodology.

 

Your teamwork score is calculated by reference to your self and peer assessment of your own demonstrated ability to work in a team.    In the self and peer evaluations you will determine your individual contribution to the team project, including planning and implementation as well as an evaluation of your ability to function effectively in a team.  Your performance will be evaluated against the following criteria:

 

·         holistically, how well you worked within the team dynamic

·         regular attendance at all team meetings

·         preparation for all team meetings

·         active participation at all team meetings

·         performance of allocated tasks within agreed time frames

·         participation in setting team goals and tasks

·         participation in achieving team goals and tasks

·         your skill at providing feedback and encouragement to other team members

·         overall contribution and involvement in planning and developing the Research Methodology

·         overall contribution and involvement in planning and developing the Teamwork Portfolio

·         overall contribution and involvement in researching and writing the Compulsory Group Assignment

 

You will also be provided with feedback from your team-mates on what they have gained most from working with you in a team, what you do particularly well and their advice to facilitate improvement of your teamwork skills. 

 

Your Teamwork Portfolio will not be separately assessed, but it must correlate with the assertions made in the self and peer evaluations.   

 

A failure by any member of a team to contribute his or her share to the researching and writing of the assignment will not be grounds for an extension or special consideration for the other members of that team.

 

 

Calculating your Teamwork Score: Your teamwork score will constitute 5% of the total marks for your assignment.  Your teamwork score takes into account:

 

·         Self-Assessment: This is your own assessment of your contribution to the team project as indicated in the Evaluation Sheet:  Self Assessment score.

·         Peer Assessment: This is your team members’ assessment of your contribution to the team project as indication in the Evaluation Sheet:  Peer Assessment score.

 

As the self and peer assessment scores may vary from student to student, it is unlikely that all team members will receive the same teamwork score.  Your self and peer assessment scores must be substantiated by the material included in your Teamwork Portfolio.  In the event of a discrepancy, the unit coordinator reserves the right to adjust your teamwork score and may request written reports from all team members on their level of contribution to any aspect of the assignment.

 

 

 

Notes

[1] The Australian Law Reform Commission's recent review of the Federal civil justice system concluded that the essential focus of legal education should be on 'what lawyers need to be able to do' rather than what lawyers may 'need to know'. Australian Law Reform Commission, Managing Justice - A Review of the Federal Justice System, (ALRC, Canberra, 1999) Report No 89, Chapter 2, 'Education, Training and Accountability' citing the American Bar Association, Legal Education and Professional Development - An Educational Continuum, (ABA, Chicago, 1992) ('MacCrate Report').

[2] See further W Cheng and M Warren, 'Making a Difference: Using peers to assess individual students' contributions to a group project' (2000) 5 (2) Teaching in Higher Education 243 at 243-244 citing (1) L S Vygotsky, 'The genesis of higher mental functioning' in J V Wertsch (Ed.), 'The Concept of Activity in Soviet Psychology', (Armonk, New York, 1981), pp144-188 and (2) L S Vygotsky, 'Mind in Society: the development of higher psychological processes', (Harvard University Press, Cambridge, 1978). See also, T Gatfield, 'Examining Student Satisfaction with Group Projects and Peer Assessment' (1999) 24 (4) Assessment and Evaluation in Higher Education 365 at 366 citing J A Mello, 'Improving individual member accountability in small group settings' (1993) 17 (2) Journal of Management Education 253.

[3] M Le Brun and R Johnstone, The Quiet (R)evolution: Improving student learning in law, (The Law Book Company Limited, Sydney, 1994), pp59-60.

[4] See further Gatfield, op cit (n3), p 366.

[5] The criteria have been developed by the Team Leader of the Second Large Grant Project, Ms Sally Kift, Assistant Dean Teaching and Learning, QUT Faculty of Law in conjunction with Dr Duncan Nulty, Higher Education Program Evaluator, QUT Teaching and Learning Support Services. See further: S Kift, 'Harnessing assessment and feedback to assure quality outcomes for graduate capability development: A legal education case study", forthcoming at Australian Association for Research in Education (AARE) Conference, Brisbane, December 2002 [http://www.aare.edu.au/index.htm (Accessed 1 November 2002)] ('Harnessing Assessment'); and QUT, Faculty of Law, 'Teaching and Learning Development Large Grants Scheme: Interim Report', (QUT, Brisbane, 2002), [https://olt.qut.edu.au/law/ASSESSMENT/sec/index.cfm?fa=displayPage&rNum=576131 (12 November 2002)].

[6] The author wishes to acknowledge the assistance of Ms Aliisa Myolanis, Policy Advisor (Learning Support), QUT Teaching and Learning Support Services in the review of the assessment model prior to its trial.

[7] Six units stated in their unit objectives that students undertaking the unit would explicitly develop their teamwork skills: LWB139 Select Issues in Torts, LWB144 Laws and Global Perspectives, LWB241 Trusts, LWB332 Commercial Law, LWB36

[7] Law of Corporate Governance and LWB434 Advanced Research and Legal Reasoning. Four of these units made no attempt to assess teamwork skill development; only the product of the teamwork was assessed. In two of those four units the teamwork exercise was optional. One unit which did not include the assessment of teamwork skill development in their unit objectives did assess teamwork skill development in a minor way: LWB142 Law Society and Justice where a small component of the assessment of a group oral presentation provided for the summative assessment of 'other skills' which may include teamwork.

[8] For example: S Stewart and B Richardson, 'Reflection and its Place in Curriculum on an Undergraduate Course: should it be assessed?' (2000) 25(4) Assessment and Evaluation in Higher Education 369; M Freeman, 'Peer assessment by groups of group work' (1995) 20 (3) Assessment and Evaluation in Higher Education 289; and D Boud, R Cohen and J Sampson, 'Peer Learning and Assessment' (1999) 24 (4) ) Assessment and Evaluation in Higher Education 413.

[9] A number of academics were also invited to address the project group to explain how they undertook the assessment of teamwork in their units.

[10] UNESCO, Distance Learning Systems and Structures: Training Manual, Report of a Sub-Regional Training Workshop, Vol. II (UNESCO, Bangkok, 1987).

[11] D Laurillard, Rethinking University Teaching: A Conversational Framework for the effective use of learning technologies ( 2nd ed, Routledge, New York, 2002), p 145.

[12] Id.

[13] Id.

[14] Ibid, p146 citing RM Palloff and K Pratt, Building Learning Communities in Cyberspace (Jossey-Bass, San Fransisco, 1999).

[15] B Collis and J Moonen, Flexible Learning in a Digital World: Experiences and Expectations (Kogan Page, London, 2001), p 9.

[16] Id.

[17] Id. See further, L Van den Brande, Flexible and Distance Learning, (John Wiley, Chichester, 1993), p xxi.

[18] Belanger, F., and Jordan, D.H., Evaluation and Implementation of Distance Learning: Technologies, Tools and Techniques (Idea Group Publishing, Hershey, 2000), pp4-5.

[19] Id.

[20] Laurillard, op cit (n 12), p146. See also Belanger and Jordan, op cit (n 19), p7.

[21] B Collis, Tele-learningin a Ditigal World: The Future of Distance Learning (International Thomson Computer Press, London, 1996), pp582-583. See also Laurillard, op cit (n 12), p146.

[22] Belanger and Jordan, op cit (n 19), p21.

[23] Ibid, p2.

[24] The value of virtual teams is explored in J Henry and M Hartzler, Tools for Virtual Teams (ASQ Quality Press, Milwaukee, 1998), pp1-10. See also D Duarte and N Snyder, Mastering Virtual Teams: Strategies, Tools and Techniques that Succeed, (Jossey-Bass Publishers, San Francisco, 2001).

[25] See further A Reynolds, 'E-Auctions: Who will protect the Consumer?', (2002) 18 J of Contract Law 75, p.75 and M.J. Radin, J.A. Rothchild, and G.M. Silverman, Internet Commerce: The Emerging Legal Framework, (Foundation Press, New York, 2002), p v.

[26] T. Hoefling, Working Virtually: Managing People for Successful Virtual Teams and Organisations, (Stylus Publishing, Virginia, 2001), p xiv.

[27] Duarte and Snyder, op cit (n 25), p9.

[28] Hoefling, op cit (n 27), p xiv.

[29] J Lipnack and J Stamps, Virtual Teams: People Working Across Boundaries with Technology (John Wiley & Sons Inc., New York, 2000), p16.

[30] Henry and Hartzler, op cit (n 25), pp 1-10. See also Duarte and Snyder, op cit (n 32), and Hoefling, op cit (n 27), p5.

[31] Duarte and Snyder, op cit (n 25), p9.

[32] Ibid, p10.

[33] Hoefling, op cit (n 27), p6.

[34] Duarte and Snyder, op cit (n 25), p 8.

[35] Id.

[36] Id.

[37] Henry and Hartzler, op cit (n 25), pp7-9.

[38] Hoeffling has offered useful tips to virtual teams in the workplace. Op cit (n 27), pp 159-161). It is submitted that Hoeffling's tips will prove particularly useful to students operating in virtual teams. Virtual tips to avoid traps 'Accept some loss of operational efficiency. This doesn't always happen but its less frustrating when a certain 'acceptable slippage' is allowed. It will be regained in other areas like speed of message delivery.' 'For some team players, who are used to being on teams and are good team players, to move to a virtual team can be a bit jolting at first. One reason that this jolt occurs is that if it's a true team with a lot of interaction, a synergy results.' This synergy may take more time to develop in a teaching and learning environment and may only arise after several virtual meetings, the team meeting at the external school or several face to face meetings. 'One problem with asynchronous communication is dissipation of group energy. The team can start to drift apart without regular contact. Creative brainstorming can fall flat if not moved along. Team leaders must help the group feel that it is together.' Where students are encouraged as part of their learning experience to rotate the leadership role, then team members share this responsibility and essentially ownership of the group 'Confront all non performance. Management responsibilities do not lessen in a virtual environment.' In a virtual learning environment it is particularly important to confront all non-communication in order to reveal whether there are any underlying concerns which need attention. 'Make a special effort to catch conflicts early and deal with them fairly. If there are miscommunications and they don't get acknowledged and remedied, it can lead to trust issues. In a virtual team environment trust issues can go unrecognized, unaddressed and unresolved far too long. In a virtual environmnet, its easier just to sweep conflict under the rug until it becomes a much bigger issue and can't be ignored anymore.'

[39] Lipnack and Stamps, op cit (n 30), p24.

[40] All unit materials are available in the study guide and on the unit's on-line teaching website [https://olt.qut.edu.au/law/LWB334/sec/index.cfm?fa=dispHomePage (Accessed 1 November 2002)].

[41] Skills are developed incrementally through three stages: Level 1 (instruction, opportunity to practice and feedback), Level 2 (advanced instruction, opportunity to practice in a simple legal scenario, feedback and reflection) and Level 3 (students should resourcefully, productively, adaptively and creatively build upon previous skill development and instruction in a more complex legal sceario). Reflection on skill development at Level 3 plays a pivotal role towards attainment. The manner in which skills have been embedded across the various years of the cirriculum ensures that students move incrementally, horizontally and vertically through each stage of skill development.

[42] Although the unit was developing teamwork skill at the highest level of undergraduate skill development (Level 3), it was anticipated that as skills had only been recently inculcated into the curriculum, a significant number of students may not have undertaken earlier units at a time when they had incorporated teamwork skill development. The materials developed for the model drew from and expanded upon materials that had been developed for students in the elective fourth year unit, LWB434 Advanced Legal Research and Reasoning.

[43] The tutorial exercises built upon tutorial exercises already being undertaken in the compulsory second year unit, LWB241 Trusts. The materials for this tutorial can be found on the on-line teaching web site for this unit [https://olt.qut.edu.au/law/LWB334/sec/index.cfm?fa=displayPage&rNum=540463 (Accessed 1 November 2002)].

[44] These discussion forums could be accessed via the unit's on-line teaching website [https://olt.qut.edu.au/law/LWB334/admin/index.cfm?fa=dispHomePage (Accessed 1 November 2002)].

[45] G Hart, Student Perspectives on the Development of Generic Capabilities at QUT: Draft Report, presented to the QUT Teaching and Learning Committee, 30 October 2001, p4.

[46] P Ramsden, Learning to Teach in Higher Education, (Routledge, London, 1992), p7; Freeman at 290; and Boud, Cohen and Sampson at 413.

[47] See L Gueldenzoph and G May, 'Collaborative Peer Evaluation: Best Practices for group member assessments' (2002) Mar Business Communication Quarterly 9.

[48] Id, D Magin, 'A Novel Technique for Comparing the reliability of Multiple Peer Assessment with that of Single Teacher Assessments of Group Process Work', (2001) 26 Assessment and Evaluation in Higher Education 139 at 140 ('A Novel Technique') citing JK Davis and S Inamdar, 'Use of Peer Ratings in Pediatric Residency' (1988) 63 Journal of Medical Education 647; F Lopez-Real and Y Chan, 'Peer Assessment of a Group Project in a Primary Mathematics Education Course' (1999) 24 Assessment and Evaluation in Higher Education 67; and Gueldenzoph and May, op cit (n 48), p9 citing T Crews and A North, 'Team Evaluation Part 2' (2000) 16 Instructional Strategies 1.

[49] This would be achieved through the implementation of self and peer evaluation, which are frequently used to overcome inequitable contribution to teamwork. See further M Lejk, M Wyvill and S Farrow, 'Group Assessment in Systems Analysis and Design: a comparison of the performance of streamed and mixed ability groups' (1999) 24 Assessment and Evaluation in Higher Education 5 at 11 citing M Lejk, M Wyvill and S Farrow, 'A survey of methods of deriving individual grades from group assessments' (1996) 21 Assessment and Evaluation in Higher Education 267. See also further Cheng and Warren, op cit (n 3), p245; R Conway, D Kember, A Sivan, and M Wu, 'Peer assessments of individual contributions to a group project' (1993) 18 (1) Assessment and Evaluation in Higher Education 45 at 46; AC Butcher, LAJ Stefani, VN Tario, 'Analysis of peer, self and staff assessment in group project work' (1995) Assessment in Education 165 at 165; J Goldfinch and R Raeside, 'Development of a peer assessment technique for obtaining individual marks on a group project' (1990) 15 (3) Assessment and Evaluation in Higher Education 210 and N Falchikov and D Magin 'Detecting gender bias in peer marking of students' group process work' (1997) Assessment and Evaluation in Higher Education 385.

[50] The unit objectives and criteria for assessment were contained in the study guide and were available on the on-line teaching web site for this unit.

[51] Lejk et al, 'Group Assessment in Systems Analysis', op cit (n 50), p11 citing M Lejk, M Wyvill and S Farrow, 'A survey of methods of deriving individual grades from group assessments' (1996) 21 Assessment and Evaluation in Higher Education 267. See also Cheng and Warren, op cit (n 3), p245; Conway, Kember, Sivan, and Wu, op cit (n 50), p46; Butcher, Stefani and Tario, op cit (n 50), p165; Goldfinch and Raeside, op cit (n 50), and Falchikov and Magin, op cit (n 50). Given the focus of this paper, further issues related to the incorporation of peer and self-assessment in the assessment model will not be explored further in this paper.

[52] These criteria are set out in Appendix 1: The Assessment Model. See further S Stewart and B Richardson, 'Reflection and its Place in Curriculum on an Undergraduate Course: should it be assessed?' (2000) 25(4) Assessment and Evaluation in Higher Education 369 at 375 citing G Brown, J Bull and M Pendlebury, Assessing Student Learning in Higher Education (London, Routledge, 1997); Ramsden, op cit (n 54), p6; P Orsmond, S Merry and K Reiling, 'The use of student derived marking criteria in peer and self-assessment' (2000) 25 (1) Assessment and Evaluation in Higher Education 23 at 23; P James, 'Blueprint for Skills Assessment in Higher Education' (2000) 25(4) Assessment and Evaluation in Higher Education 353 at 363.

[53] Ramsden, op cit (n 47), p6.

[54] Orsmond, Merry and Reiling, op cit (n 53), p 23.

[55] Stewart and Richardson, op cit (n 53), pp 3

[55] and 370.

[56] B O'Donovan, M Price and C Rust, 'The student experience of criterion-referenced assessment (through the introduction of a common assessment grid)' (2000) 38 Innovations in Education and Teaching International 71 at 79 citing CV Gipps, Beyond Testing (London, The Farmer Press, 1994).

[57] Stewart and Richardson, op cit (n 53), p375 citing J Chapman, 'Agonising about Assessment' in D Fish and C Cole (eds) Developing Professional Judgement in Health Care (Oxford, Butterworth-Heinemann, 1988).

[58] The assessment criteria are set out in Appendix 1: The Assessment Model

[59] The self and peer evaluation instrument can be accessed on the unit's on-line teaching website, op cit (see n 45).

[60] Internal students undertook this exercise in class. See further S Hanrahan and G Isaacs, 'Assessing Self and Peer-assessment: the students' views' (2001) 20(1) Higher Education Research and Development 53 at p54-55; and O'Donovan et al, op cit (n 57), p80.

[61] Orsmond et al, op cit (n 53), pp33-34.

[62] O'Donovan et al, op cit (n 57), p80 citing DR Sadler, 'Specifying and promulgating acheivement standards' (1987) 13 Oxford Review of Education 191.

[63] See generally, Stewart and Richardson, op cit (n 53).

[64] Units using these reflections were LWB434 Advanced Legal Research and LWB241 Trusts. The reflection sheets used in the assessment model are available on the on-line teaching site for the unit, op sit (n 45).

[65] Lejk et al, 'Group Assessment in Systems Analysis', op cit (n 50), p7, reported that this view had been supported by student feedback. Gibbs concurs. See G Gibbs, Learning in Teams: A Tutor Guide, (Oxford Centre for Staff Development, Oxford, 1995), p8 ('Learning in Teams'). Lejk et al, 'Group Assessment in Systems Analysis', op cit (n 50), p7 and 11 citing G Gibbs, Assessing Student Centred Courses (Oxford, Oxford Centre for Staff Development, 1995).

[66] Lejk et al, 'Group Assessment in Systems Analysis', op cit (n 50), pp 7 and 11 citing G Gibbs, Assessing Student Centred Courses (Oxford, Oxford Centre for Staff Development, 1995).

[67] Hart, op cit (n 46), pp5-6.

[68] Id.

[69] Lejk et al, 'Group Assessment in Systems Analysis', op cit (n 50), pp 8-9 and 11.

[70] Id.

[71] See further Lejk et al, 'Group Assessment in Systems Analysis', op cit (n 50), pp7-11.

[72] Id.

[73] For reasons of practicality, internal students requiring placement were placed in teams with other students from their tutorial groups.

[74] Gibbs, Learning in Teams, op cit (n 66), p8.

[75] The author wishes to acknowledge the valuable assistance of Dr Duncan Nulty, Higher Education Program Evaluation, QUT Teaching and Learning Support Services in the development of the survey.

[76] If the sample is skewed at all, it is towards students who are able to attend on campus.

[77] Although the responses to these questions indicate a significant neutral response, the results are considered indicative of the trend in student opinion. This is reinforced by the low negative response.

[78] This was illustrated in Tables 1 and 2.

[79] See further Table 4.

[80] Due to a number of students withdrawing from the unit, 8 teams were reduced to teams of 2.

[81] One staff member remarked that the assignments were of a much higher quality those submitted by students individually in an earlier offering of the unit.

[82] Cheng and Warren, op cit (n 3), p244 citing G Gibbs, S Habeshaw and T Habeshaw, Interesting ways to assess your students, (Technical and Education Services Ltd, Bristol, 1986)

[83] A quantitative analysis of results in this assignment is explored below.

[84] External students will undertake an exercise at the external school.