Legal Education Digest
S Field and L Jones
Law Teacher, Vol 44, No. 3, 2010, pp 378-390
Students are in general open to new forms of assessment including the use of e-learning but are concerned with the fairness of assessment and the level of feedback they receive. Moreover, research indicates that computer based assessments have the potential to provide an effective mechanism for giving feedback and are, at least, moderately effective in supporting student learning. We sought to investigate whether our innovations were actually improving student learning. We wished to focus on the benefits and the outcomes from using technology to support learning in the law, both in terms of supporting learners – through greater opportunity for individual formative feedback – but without imposing greater burdens on ever-beleaguered colleagues.
There were also other issues underpinning our research, such as practical issues, ie. limits on resources (on lecturers’ time, and also at the institutional level), and ethical issues, such as the growing unease amongst colleagues about the incidence and prevalence of plagiarism in coursework.
As Gerdy has found, ‘Student plagiarism occurs despite the facts that the students themselves know that such conduct is wrong’. Essay mills have evolved and become more sophisticated, and although most claim that they only provide ‘model papers’, research indicates that students hand these in unaltered as their own assignments. Moreover, plagiarism detection software may be able to scan web pages but cannot scan books or search material on subscription databases such as such as Westlaw or Lexis. Similar concerns have been voiced by Higher Education Funding Council for England (HEFCE). In its 2009 Report of the Sub-Committee on Teaching Quality and the Student Experience, HEFCE point to ‘more coursework, group work and continuous assessment’ as key factors in the rise in plagiarism in higher education.
One concern was whether a move to assessment by 100 per cent examination would be a progressive, or retrospective step in terms of assessment strategy. Williams has argued eloquently that the closed book final exam ‘has become an anachronism’. He believes that is does not assess ‘deep conceptual understanding’, relying as it does on ‘cramming’ the night before the examination, and ‘data pumping on the day. We aimed to investigate to what extent a move to assessment by 100 per cent ‘seen’ examination would constitute a successful halfway house between the unseen examination and coursework, thereby addressing these concerns, and how blended learning could support this innovative strategy.
The aims of the LLB program at the University of Brighton are to provide an interesting, stimulating and relevant program and to develop students’ academic, practical and transferable skills. Students are provided with solid academic grounding in the recognised foundations of law subjects together with other key aspects of business law and knowledge of internal and external issues relating to the operation of businesses.
In looking closely at assessment on the LLB program it is clear that the majority of core law modules (particularly in years one and two) are summatively assessed on a 70 per cent ‘unseen’ examination with 30 per cent coursework ratio. However there are exceptions: the second year Legal Case Study Moot is assessed by written coursework 40 per cent and an oral examination (moot) 60 per cent. The final year Property Law module is assessed by 30 per cent course work and 70 per cent as a ‘part seen’ examination (a detailed scenario is released to students two weeks prior to the examination date but questions on the scenario are unseen).
In the academic year 2007/8 the law team agreed to introduce a new and innovative form of assessment, namely the 100 per cent seen examination. The practical rationale for considering the introduction of core modules assessed by 100 per cent examination was partly to reduce the burden of summative assessment throughout the academic year which impacted upon student seminar preparation and student learning. The decision was also motivated by broader, ethical concerns about plagiarism in coursework as well as a drive towards new and innovative assessment strategies. This assessment innovation was to be trialled in one of the core second year modules, Criminal Law.
The three hour Criminal Law examination paper is released to students two weeks prior to the examination date and takes the form of two compulsory problem questions which encompass multiple topic areas covered in the module, and a choice of two out of four essay questions. It was envisaged that this format would create what Williams has termed an ‘authentic assessment’ item, with an emphasis on the importance of analysis rather than content knowledge, and a focus on the skills students have acquired throughout the academic year and with questions designed to be very specific to the module material in order to thwart the cheat sites.
The global context in which education takes place formed a backdrop to our project since the introduction of 100 per cent examinations addressed concerns about ethical practice (particularly with regard to coursework). Moreover, it was felt that the introduction of a seen examination addressed concerns about a return to the traditional 100 per cent unseen examination which may have been viewed as a retrogressive step in student assessment.
This research builds on the ‘HEFCE strategy for e-learning’, focusing on enhancing learning and teaching through the use of technology. Blended learning ensures that online communication is not faceless but if it is to enhance the student experience it is essential to get the balance right and to have the support of the students on the module. The aim of our research was to assess the advantages and disadvantages of introducing elements of e-learning into formalised legal teaching as a means of offering additional opportunities for formative feedback, particularly where innovative assessment practices had been introduced.
It is well established that good assessment goes to the heart of the student learning experience: the Quality Assurance Agency notes that feedback should ‘promote learning and facilitate improvement’, and Nicol and Macfarlane-Dick argue that ‘feedback’ and ‘feed-forward’ should be systematically embedded in curriculum practices. They conclude that ‘creating an environment rich with useful, high-quality feedback that supports effective student learning is possible without a negative impact on staff time’. This thinking was essential to our research.
At the University of Brighton there are approximately 80 students in each year of the LLB (Hons) Law with Business Degree.
While increasingly used in many fields, blended learning had not been used in the field of law at the University. Before introducing a series of blended learning into LLB (Hons) Law with Business degree and investigating its use within a law environment, a pilot study was devised and as the results from the pilot study proved encouraging, blended learning was introduced into the degree programme in 2008-9.
In the Spring term of the academic year 2007-8 e-learning activities (a set of revision materials) were introduced into the second year LLB (Hons) Law with Business Criminal Law module (one of the Foundation of Law modules undertaken by all qualifying law degree students). Student participation was monitored (through statistical analysis) and a student feedback exercise conducted (a questionnaire with both closed-ended and open-ended questions). The materials comprised two sets of revision aids, four multiple choice revision tests (answers were also supplied separately), as well as a mock examination question, which students were advised to attempt under exam conditions and then send to the lecturer for grading by a specified date; they were also told that if they failed to meet the submission deadline, they would nonetheless be able to access a full (three page) written solution to the question (posted at a later date on the University of Brighton’s virtual learning environment, ‘studentcentral’).
Of the 68 students registered on the module, 64 accessed the materials. The majority of students accessed the materials on more than one occasion and of those students, 29 submitted an answer to the mock exam question for grading. Feedback received was generally positive: of the 45 students who completed the questionnaire, 43 felt that the e-learning session had been a positive learning experience.
Following on from the success of the pilot study, it was decided to introduce blended learning into two of the Foundations of Law modules; one in year one (Public Law) and one in year two (Criminal Law). It was felt that the gradual transition over two years to the new format would enable students to become familiar with the new e-learning platform. Thus the Public Law module served as a means of introducing the learning technologies to the students, while the Criminal Law module formed the basis for our research into the role of blended learning in a module assessed by means of 100 per cent seen examination paper.
The research project was completed in May 2010. This paper will present the findings from the study in terms of student engagement, student perceptions, and reflections on the whole process.
The medium and design of course material is one of the most crucial aspects of blended learning. As Ozkan and Koseler found, ‘Content quality depends on how well the e-learning environment is designed and managed’. Moreover, it has been well documented that learners place great value on content which is ‘well organised, effectively presented ... clearly written, the right length, useful, [and] flexible’. Thus in both modules there are a broad range of activities, from multiple choice tests, to short answers on topics covered in lectures, mini research questions and formatively assessed full length exam-style essay and problem questions. This approach increased student autonomy and encouraged the students to take ownership of their learning experience. Moreover, students were given independence to learn at a comfortable speed.
The e-learning sessions were supported by face to face seminars. The students were generally required to have completed certain tasks and thereby acquired a certain level of knowledge for face to face sessions. Although this approach curtailed their autonomy somewhat, this strategy was adopted in order to mitigate the likelihood of students falling too far behind.
An important point to note is that since the e-learning sessions replace face to face seminars, no additional burdens are placed on the lecturer in terms of time commitment/resources: instead of holding face to face seminars, the lecturer is ‘freed up’ to mark the online assignments, or engage with the students in a virtual learning environment.
In order to monitor student access of the materials a statistical tracker system on ‘studentcentral’ (Blackboard) was enabled over the two year period. The statistical information provided established that the majority of students registered on the modules had engaged with the materials.
In the first year (2008/09) 84 per cent had done so; encouragingly, this figure had increased – to 89 per cent – by the second year of the research period.
The use of an online teaching and learning strategy per se was completely new for these students. First year students were new to the course and whilst second year students were familiar with ‘studentcentral’ as a valuable online resource, they had only used it to access module materials which were supplementary to their weekly face-to-face lectures and seminars, or to access teaching materials, such as lecture notes, that they had missed due to absence; none had any experience of online learning in lieu of face-to-face sessions. Moreover, several researchers consider learners’ perceived effectiveness as an important indicator of an effective teaching and learning model. For this reason, a student feedback exercise to determine students’ perceptions of this e-learning activity was felt to be an essential part of the investigation into whether – and how – online learning might enhance their learning experience.
The instruments used for this exercise were short questionnaires with closed-ended and open-ended questions based on the questionnaires used for the pilot study.
The questionnaires were completed anonymously to encourage students to be as frank and candid as possible about their views. The response rate was approximately 56 per cent. In addition, at the end of the academic year 2009/10, an additional feedback exercise – a questionnaire with open ended questions – was conducted with the Criminal Law students in order to enable reflection on the new platform at the end of the two year research period.
The responses to the questionnaire were analysed using the Statistical Package for the Social Sciences (SPSS) software programme.
It was interesting to note that the data from the survey suggest an increased level of engagement with the module than had been the case under the older models of delivery.
Approximately 98 per cent of the students completing the questionnaires had participated in the e-learning activities. From our analysis of the data gained through the tracking system on ‘studentcentral’ it was clear that the majority of students attempting the e-learning activities completed at least 75 per cent of all activities with more than half of the total cohort completing all the activities. The percentage of students who stated that they always or even usually prepared for traditional seminars fell below that of those who prepared for and participated in the e-learning sessions.
Students appeared divided as to whether the e-learning sessions had helped in their understanding of the subject matter, although least half of the students agreed that it had assisted.
What was clear from the results is that the students enjoyed the flexibility of the e-learning sessions, appreciating being able to choose when and where to complete the material.
However, it was interesting to note (from the tracking system on ‘studentcentral’) that – in spite of these views – the majority of students (74 per cent) seemed to undertake their e-learning work during what would have been normal face-to-face seminar times.
The majority of students agreed that working with e-learning material developed their skills as independent learners.
In the first year of the research project students seemed less confident of the benefits of e-learning as an aid for summative assessment preparation. However, the questionnaires were completed before the examinations that year, and by the second year of the research, having experienced e-learning in their first year, students appeared to feel more confident in the benefits of the formative assessment opportunities offered by the e-learning platform.
There were also a considerable number of references by students, at the end of the second year of the research study, to the benefits the new platform provided in terms of opportunities for formative assessment and feedback.
Interestingly, some of the students who participated in this research project indicated that until feedback is received from the tutor they are unsure that they are reading the materials in the ‘right’ way or that the answers they are drafting for seminars are ‘correct’.
Prior to the introduction of the new format formative assessment opportunities did exist, but with fairly low take-up, and for many students the only ‘real’ feedback they received was in relation to formal coursework submitted after several weeks of study.
The results of this research project demonstrate that in general students are prepared to participate in formative assessment via e-learning, and statistically more likely to engage with the materials than they are to prepare for all face to face seminars. Students clearly appreciate the flexibility and accessibility of e-learning. However their acceptance and perception of blended learning as an effective learning tool for developing understanding of topic areas and as preparation for summative assessment is more undetermined. This complements the findings of Miller that students perceive feedback from computer based assessment to be moderately effective in supporting their learning.
Moreover, our initial findings add support to the view expressed by Nicol and Macfarlane-Dick that it is possible for lecturers to give feedback to support learning without a negative impact on their time. It appears that blended learning may well provide the means to do this: it has enhanced the learning experience of the majority of students taking the modules without the need for additional resources. Thus, the practical benefits offered by blended learning can be said to operate at two different levels, namely in terms of efficiency on the part of the institution and enhancement in terms of the student learning experience.
Finally, our project demonstrates that blended learning is capable of supporting shifts in assessment practices: a renewed approach to assessment by examination may find resonance in a sector increasingly concerned over the ethics of coursework and a move to a seen examination appears to work well as a halfway house between assessment by coursework and the traditional unseen examination.