Re-imagining exams: How do assessment adjustments impact on inclusion?
Joanna Tai1, Rola Ajjawi1, Margaret Bearman1, Joanne Dargusch2, Mary Dracup1, Lois Harris2, Paige Mahoney1
Executive summary
Exams and other high-stakes time-limited assessments can act as barriers to success for some students in higher education. Though required by Australian law, adjustments do not necessarily lead to equitable academic outcomes (Brett, 2016; Kilpatrick et al., 2017), nor do they always address students’ actual access requirements (Waterfield & West, 2006). A system which focuses only on making reactive accommodations is likely to become overwhelmed as diverse students increasingly participate in higher education. Rather than continuing to view disability as a problem to address at an individual level, a shift to focus on inclusive assessment design may also hold promise for a broader range of diverse students.
In addition to students with a disability (SWD), in recent years students across equity groups, including regional, rural, and remote (RRR) and/or low socioeconomic status (SES) (Koshy, 2019), are increasingly participating in higher education. Social inclusion therefore becomes a concern when considering what happens in assessment practices. There are also groups such as First in Family (FiF) students who have not been formally designated an equity group by the government, yet due to intersecting demographic, social and cultural characteristics, share similar experiences to other equity groups within higher education (O’Shea, 2016). There is a need to better understand how these types of equity markers intersect to compound disadvantage (Drury & Charles, 2016; Nelson et al., 2017). This is highly important within the context of assessment, since assessment has a substantial impact on success and retention for all students, but particularly those with intersecting equity group memberships (Ajjawi et al., 2020).
The project
This research project therefore aimed to 1) understand SWDs’ experiences of exams, and 2) transform exam design and practice to be more inclusive. It focused on high-stakes, time-limited assessments: traditional examinations and their ilk. It considered the impact of disability on students’ exam experiences and also considered effects relating to their intersecting identities, with particular attention on the effects of RRR, FiF, and/or low SES backgrounds. With a multi-disciplinary team spanning two universities, researchers sought to identify how adjustments impacted on inclusion as perceived and experienced by SWDs. A series of participatory workshops were then used to explore what could be done to re-imagine exams. By recasting high-stakes assessments as contextualised practices where people and materials interact in social environments, insight was gained into how disadvantage occurs through these practices for SWDs, and how challenges are compounded where SWDs are also RRR, FiF and/or low SES.
Method
A two-phase research design was used to achieve the two aims. The first phase invited SWDs to share their experiences of exams through interviews, taking into account the complexity of their individual circumstances. Across two universities, 40 students participated in interviews and 11 additional students contributed written or audio responses to the interview prompts. The second phase invited assessment stakeholders (academics, accessibility staff, and students) to participate in workshops, grappling with aspects of exam design to identify what could be modified to improve inclusion in four specific units (i.e. modules or subjects) of study, two at each university.
Project findings overall suggest that, while most students had experiences that were not inclusive in relation to their high-stakes timed assessment, there was no single “easy” solution to re-imagining exams, with a combination of approaches required. From the student interviews, patterns were identified in terms of which aspects could be improved. Staff support, including the development of relationships, was powerful in ensuring students felt included. Minimising the bureaucracy required to obtain adjustments was spoken of positively. Within the context of Covid-19, the places and spaces of exams shifted to the home environment and, while this was generally seen as reducing the need for some types of adjustments relating to equipment, furniture and separate exam spaces, some students spoke of disruptive home environments with little space for study-related requirements. Exam design was a key factor in students’ experience, including time and timing, format, and authenticity. The complexity of individual circumstances created a different combination of considerations in each case, and this is demonstrated in further detail through a student case study.
Participatory workshops were used to explore what was possible both short-term and longer-term to re-imagine exams. While narratives from the first phase of the study were included as part of the workshops, contributions from student workshop participants were particularly valued. A shift to a broader understanding of inclusion underpinned actions to improve student-staff interactions, communication between stakeholders, and assessment arrangements. Changes made possible due to Covid-19 were also recognised as contributing to inclusive assessment. From the workshop series, four unit case studies are presented, outlining the context, the assessments under review, and potential, implemented and future planned changes. While formally investigating the impact of these changes is beyond the scope of this project, where unit chairs/coordinators (UCs) were able to implement change during the life of the project, these changes have been perceived positively by students. The most valued aspect of the workshops was that there was a series of opportunities to come together to discuss dilemmas, with input from a wide variety of stakeholders with different perspectives.
Key findings
Key messages that arise from the project findings are:
- Inclusion is an ongoing and proactive process which needs to be continually enacted by all stakeholders in assessment.
- Assessment design must balance the ideal and the pragmatic in context: there is no single solution which will work for all situations.
- A coordinated and comprehensive approach to inclusive assessment design is required.
- We must create opportunities to listen, discuss, and collaboratively problem-solve.
The findings of this work have led to the development of a range of resources which can be taken up beyond the project: specific advice relating to exams; an inclusive assessment framework for designing assessment; guidance for universities; and workshop resources to effect change. These are available from the project website.
Read the full report: Re-imagining exams: How do assessment adjustments impact on inclusion?
This research was conducted under the NCSEHE Research Grants Program, funded by the Australian Government Department of Education, Skills and Employment.
1Deakin University
2CQUniversity
Expert review
Professor Michael Henderson
Director, Educational Design and Innovation
Faculty of Education, Monash University
In this report, JoannaTai and colleagues Rola Ajjawi, Margaret Bearman, Joanne Dargusch, Mary Dracup, Lois Harris and Paige Mahoney make a compelling case for proactive assessment designs to better support broad goals of equity and inclusion.
The authors found that inclusion cannot be resolved easily, and often results in a compromise between the demands of the ideal and pragmatic. They argue that inclusion requires an ongoing commitment as well as a coordinated and comprehensive approach. They also conclude that, in order to find proactive solutions, there is the need to “create opportunities to listen, discuss and collaboratively problem-solve”.
The report offers valuable advice and strategies in creating these opportunities. In addition, I was particularly taken by the authors’ casual observation that the very act of seeking input from a wide range of stakeholders including educators, administrators and learners to identify needs and discuss dilemmas was in itself a valued and agentic process.
Without recounting the specific findings or advice offered in the report, it is perhaps more useful to note that current common practices such as assessment adjustments – while motivated by inclusive goals – are not always successful. Instead of pursuing this retroactive tradition the authors drew on a sociomaterial perspective and participatory workshops to gain new insights about the experiences, challenges and needs of stakeholders. What is particularly valuable about this approach was the authors’ intersectional approach to understanding disadvantage, including: students with disability; regional, rural and remote; and student from low socioeconomic backgrounds.
The report provides valuable and actionable advice to all educational institutions and authorities.