AustLII Home | Databases | WorldLII | Search | Feedback

Legal Education Digest

Legal Education Digest
You are here:  AustLII >> Databases >> Legal Education Digest >> 2011 >> [2011] LegEdDig 44

Database Search | Name Search | Recent Articles | Noteup | LawCite | Author Info | Download | Help

Collins, P et al --- "Rocky rhetoric and hard reality: the academic's dilemma surrounding assessment" [2011] LegEdDig 44; (2011) 19(3) Legal Education Digest 46


Rocky rhetoric and hard reality: the academic’s dilemma surrounding assessment

P Collins, T Brackin and C Hart

Legal Education Review, Vol. 20, 2010, pp 157-192

The Bradley Review of Higher Education states that both more money and more students from diverse backgrounds will be necessary for Australia to effectively compete in the global market. Assessment is a factor which, as a learning driver for students, must figure strongly in any consideration of the agenda to widen participation in higher education. In this regard, we have to closely consider the role of assessment within the institutional context. A focus on improving educational outcomes surely needs to seriously consider the role of assessment and factors that may prevent the outcomes hoped for in the Bradley Review. The Government response has been the establishment of the Tertiary Education Quality and Standards Agency (‘TESQA’) and budgetary responses focusing on equity, access, sustainability, research and quality teaching.

The drive to increase numbers of students aged 25-34 to 40 per cent by 2020 includes encouraging students who would previously not have considered university education. Therefore, students from diverse backgrounds, including students who are the first in their family to study at university (‘first in family’), from regional and remote areas, Indigenous communities and low socioeconomic groups (up to 20 per cent) become the focus in terms of student recruitment. With this comes the correlated need to retain and progress these students to produce ‘quality’ and engaged students who are ‘work ready’ at the end of the education process. For law schools, this requires adopting a legal practice orientation and a broader curriculum, to engage a more diverse student body catering for differing career options for an uncertain future.

It is well known that student learning is driven by assessment and is the most important factor when it comes to determining whether the student has reached a satisfactory standard. Student experience of assessment and thus their attitude to learning is also a vital component in students’ evaluation of both academics and the institution. However, the drive for assessment of learning, as part of the public accountability requirements, can lose sight of the need to balance assessment for learning which is essential if students are to improve their skills and graduate as successful contributors to their chosen field.

The research aim was to gather and analyse data in the formative years (for learners, teachers and program designers) of an undergraduate law program. The question of what these factors are and how they influence assessment needs to be understood if quality education and, therefore, quality graduates are to be produced.

The study focused on assessment practice: how academics approach assessment from their personal perspective; what motivates the creation of particular assessments; and issues surrounding the actual assessment of students. It looked at internal factors such as academics’ perceptions of how students approach assessment and the role of skills and graduate attributes in the assessment process, along with external considerations such as the impact on assessment of the academics’ work environment. The academics surveyed are all course leaders in first- and second-year core undergraduate curriculum courses in the LLB programme and they work across two campuses. The three authors/researchers were included as instruments in their own research, as they are also core course leaders, along with the other 10 academics involved.

A grounded theory approach was adopted using a qualitative research method which involved (1) a pre-interview survey of the academics; (2) a half-hour semi-structured interview; (3) qualitative information obtained from course evaluations; and (4) a student focus group.

This approach employs the ability to observe and compare participant’s data, to determine what is happening for the person in their domain and to draw tentative conclusions where possible rather than testing a specific hypothesis.

The survey and interview demonstrated that factors impacting on academics – such as, workload demands, research output requirements and other demands including student retention and progression – affect their assessment design and thus, ultimately, the production of quality graduates. With many of the traditional forms of content-based assessment – such as, exams and essays based on standard-form problems, which are predictable and easily administered – a superficial learning known as subject-based learning occurs. These assessments are in the main assessments of learning, which ensure students have understood the material they have been presented with, rather than assessments for learning.

However, some academics go further and require assessment to be for learning, where the assessment task itself is part of the learning process and students are required to demonstrate skills needed for later use in the workforce as well as discipline knowledge.

The skills identified collectively by the law school as appropriate for students include discipline-based knowledge and key skills for lawyers – such as ethical research and inquiry; problem-solving ability; academic and professional literacy; written and oral communication; interpersonal skills; teamwork; cultural literacy; management, planning and organisational skills; creativity, initiative and enterprise; sustainable practice; and doctrinal knowledge.

Some academics acknowledged that assessment should develop the graduate attributes that link with the ideal skills for a student in their professional practice; in fact, giving assessment a ‘double duty’.

The overarching graduate attributes (as opposed to specific graduate skills) at the institutional level where this research was undertaken are identified as: discipline expertise, professional practice, global citizenship, scholarship and lifelong learning skills.

The implementation of graduate attributes can be problematic when some academics seem to feel a lack of ownership or affinity with the types of graduate attributes promoted by the institution.

Studies indicate that, to engage students, assessment must provide some or all of the following: opportunities for ‘active’ learning; collaboration between students; tasks that are perceived by students to be authentic; opportunities for student reflection; and opportunities for students to manage their own learning processes.

Engaging assessments that include skills often require team collaboration; oral presentations; and use of portfolios or journals of reflection on the problem-solving relating to course content, involving application of theory to practical examples.

However, these types of assessment require more commitment of staff time and resources. In a climate in which academics are being asked to restrict their face-to-face contact time and cut down on casual tutors and marking contracts, the needs of the student and the demands of the institution leave the academic who cares between a rock and a hard place:

With universities in the current climate moving to more restrictions on academics, the ability to ‘play’ and access the thinking and creative time needed for genuine engagement is diminishing. Paula Baron notes a heavy burden can be placed on an academics’ time with ‘the necessity to teach skills as well as content in courses; and the tendency to move away from doctrinal scholarship to the comparative, theoretical and sociological exploration of law’. Despite the fact that teaching and assessing skills requires perhaps greater resources and time commitments from academics, many still acknowledge that it is an important aspect of teaching in a professional degree qualification.

Many academics find that the demands placed on them do not align and so create uncertainties that, with a lack of support, are encouraging some to retire or leave the profession. In a professional degree qualification such as the LLB, there is an obligation on academics not only to the institution of the university but also as officers of the court. Law academics have a professional obligation to uphold the standards of the profession and make sure that graduates will be able to carry out their work to a professional standard. This creates a tension when academics are being required to achieve a certain level of progression and retention of students, taking into account the demand for an increase of 40 per cent in student numbers, including students from diverse backgrounds.

The degree to which academics can meet the new demands in quality learning outcomes for graduate attributes and skills, the demands of the profession and the demands of the higher education institution for retention and progression can be seen to be placing conflicting loyalties on academics. It is clear that these institutional constraints and directions are external factors which have an impact on how an academic assesses students.

Our research identified some other areas of the assessment process that lack clarity, leading to uncertainty for academics when assessing and indicating a need for further research and refinement. These factors include the academic skills needed, and the process that occurs, at the micro level of actual assessment; the contribution and significance of technology in assisting, or otherwise, with assessment; and the role evaluations have in compelling certain assessment outcomes.

Implementing and assessing skills can be seen as an uncertain art, particularly in a higher education environment that imposes ‘retention and progression’ requirements on academics, which can set up conflicting demands.

How an academic actually assesses the level of skill attainment seems heavily reliant on the academic’s own judgement, which is in turn based on their experience.

While the art of assessing skills and graduate attributes would appear, from both our research and from the literature, to be uncertain, the need for academics to grapple with the professional, institutional and government demands to retain and progress an increasing number of diversified students to achieve ‘quality’ graduates remains. This has led to the use of tools such as technology – which some consider a panacea and others an added cause for stress, ultimately affecting assessment and student learning.

With the higher education sector competing in an open globalised market, together with encouragement for increased attendance from diversified student populations, many universities have responded in a similar vein in their organisational changes and uptake of IT. With a diversified market, including mature-aged students looking for flexible lifelong learning opportunities to prepare them for uncertain futures, there is a drive for virtual education to prepare quality students for a digitalised world. Students are given assessments incorporating digital literacy skills requiring development of self-directed learning styles with the teaching style changing to a facilitative mentoring mode.

With distance education, students’ evaluation of the academic and their course often occurs through their perception of the teacher’s engagement with the student online and the provision of interactive and Internet-based resources. For older academics, this is an extra burden as they have to adapt and learn the new technologies, becoming an expert in order to provide interactive and educational virtual environments that can incorporate knowledge and skill training to produce the best graduates.

Another concern raised in the research and closely linked to the pressures from the current higher education environment is the evaluation of teaching and its impact on assessment practices. As mentioned at the outset, the push for ‘quality’ graduates means that the systemised language of accountability and number crunching comes into play with a plethora of testing instruments being introduced. Trust is no longer placed in the academic to be professional in their assessment and grading of the student. More quality assurance testing has to occur to justify government expenditure of tax payers’ funds and to work as a carrot type incentive to inspire academics to ensure they produce ‘quality’ graduates. Further, the student is often placed in the role of evaluator, and it is questionable just how well-placed they are to make this judgement. Stress factors increase when the student evaluations are perceived to arise from poorly designed evaluation systems with perceptions that the highest ratings are based on ‘the entertainment value and ease of the course’. Much of the ‘quality’ assurance is largely dependent on the capability of the testing instrument to genuinely assess the graduates’ ‘quality’ or the teaching and learning outcomes.

It seems that, while higher education is being induced to diversify its intake and accommodate difference, little is being done to acknowledge these differences in the evaluative tests, such as looking at cultural and socioeconomic factors. Progression rates of students to graduations seemed to have improved. However, understanding the reasons for this is not so clear: it could be new and improved teaching; better or more motivated students; or lower standards and pass rates.

The terms ‘inputs’ and ‘outcomes’ are frequently used in discussion, and there seems to be a growing move towards assessing outcomes rather than inputs, which understandably are seen as less valid indicators of standards.

Richard Johnstone and Sumitra Vignaendra’s thorough study reflects an attitude by law academics that student evaluations are often used in staff reviews ‘as a ‘stick’ to ‘manage’ teaching quality in schools’ which only ‘encouraged teachers to stick to tried and trusted methods such as straight lecturing and other forms of ‘spoon-feeding’ in order to ensure that their student ‘ratings’ were high enough to ensure their tenure or promotion’.

The different models of evaluation used at the course level, program level, institutional and national level can have variations in their theoretical underpinnings and design and ultimately impact on the approach an academic takes to assessment.

The solution to these interconnected dilemmas — namely, the institutional demands such as workload, student retention/progression and research output; professional skill requirements and admission hurdles; and government demands for ‘first in family’, low socioeconomic, rural and remote students to become quality graduates — may well be to focus on changes to the environment and context in which academics construct assessment. The range of skills now extends across many capabilities; such as, facilitator; technology expert; designer; manager/administrator; advisor/counselor; mentor; assessor; and researcher.

Studies on academic satisfaction indicate clearly what is needed to improve alarming imbalances in work–life experience and stress and depression levels. However, little response seems to have occurred.

The research undertaken identified issues such as different values impacting the perceived purpose of assessment; what graduate attributes and skills, if any, should be assessed; the impact of institutional and professional demands on what assessments academics will set; and how these determine the types of assessments utilised. This study has demonstrated that there is a variety of approaches in response to the higher education context based on academics’ individual values. However, these responses are within a consistent range being presented in the literature. This reinforces the need to address these concerns to improve assessment and thus the quality of graduates.

Despite the hope, there is still a risk of widening the divide that exists between the rhetoric and the reality. Much of the bridging of this divide has been dependent on the good will of committed academics who value their professional teaching and research enough to keep batting against the odds. Do we want to be consumed by ‘the greedy university’ or do we reject the long hours and settle for less promotion and job satisfaction? These are real questions academics are now being forced to face as they are caught between the rocky rhetoric and the hard reality. The impact of these concerns on the focal point of assessment highlights the need for further consideration of the very real way in which the ‘rhetoric’ of higher education and the ‘reality’ at the coal face is perceived by those in it, thus effecting the ultimate product of higher education – the ‘quality’ graduate.


AustLII: Copyright Policy | Disclaimers | Privacy Policy | Feedback
URL: http://www.austlii.edu.au/au/journals/LegEdDig/2011/44.html