Research update
Re-imagining exams: How do assessment adjustments impact on inclusion?
One of 17 projects funded under the sixth round of the NCSEHE Research Grants Program, this Deakin University-led research is investigating barriers and enablers to inclusive assessment design.
This month, Chief Investigator Dr Joanna Tai from the Centre for Research in Assessment and Digital Learning (CRADLE) provides a progress update, including preliminary findings from the research to date.
Project summary
High-stakes, timed assessments can be problematic for students with disabilities. Though assessment adjustments are enshrined in legislation, equitable outcomes are not assured, and evidence for their efficacy is unclear. Students with disabilities’ membership of additional equity groups may contribute to disadvantage which cannot solely be addressed through assessment adjustments. This project shifts the focus to understanding the broader context of, and barriers and enablers to, inclusive assessment design. It will use student voice data and workshops with staff and students to research inclusive assessment development. This will inform an inclusive assessment framework and underpin recommendations for policy and practice.
Research activities and preliminary findings
Pre-existing survey data analysis
Qualitative analysis was conducted on the pre-existing April 2020 survey data contributed by 38 returning (i.e. not first year) students registered with the Disability Resource Centre at Deakin University.
Assessments discussed included exams, quizzes, assignments, group work, oral presentations, and placements. While each assessment type had its downsides, students also reported features of each that allowed them to demonstrate their learning. Students reported a range of positive and negative assessment experiences:
- Positive experiences stemmed from the willingness of both academic and disability support staff to accommodate their needs without question.
- Negative experiences were related to additional efforts required to secure adjustments, and a lack of flexibility in assessment.
- Adjustments relating to assessment were valued, especially extensions of time. Some students reported not knowing the possibilities for these adjustments when they first started at Deakin.
Aspects of suggested inclusive assessment design, directly drawn from survey respondents’ data, can be broken down into three areas:
- assessment sequencing within a unit of study (to support development of student work over time)
- assessment choice (for example, negotiating the format or topic of assessment within set parameters)
- assessment format (for example, oral presentations being pre-recorded).
Feedback design was also seen as important to supportive inclusive assessment.
Student narrative interviews
We received an overwhelming response from students willing to share their experiences via interview. While we had proposed to interview 15 students at CQUniversity and Deakin each, we increased this amount to 20 at each institution, and also submitted an ethics amendment to allow students to submit a written/recorded response to prompts in lieu of an interview.
A total of 40 interviews, 10 written and one recorded response comprise the dataset for this phase, with 24 of 40 interviewed students being rural, regional or remote, and 24 of 40 from a low socioeconomic status (SES) background or the first in their family at university.
We have taken a pragmatic approach to initial data analysis in anticipation of the workshops, identifying 1) the overall experience of students, and 2) the social and material aspects of exams which contribute to student experiences.
- The impact of COVID-19-related restrictions has been significant, as students found themselves sitting exams at home, which were largely not invigilated/proctored, and/or had more “assignment” type assessments. This had benefits, as blanket time extensions were provided to account for any potential issues.
- Many students felt more comfortable at home with the support of friends/family, did not have to endure exhausting travel to an exam centre, and were able to take breaks as required. However, issues with technology did provoke additional anxiety, and some students found it more difficult to concentrate outside of the traditional, supervised exam environment.
- In terms of social aspects of exams, students expressed the importance of relationships and academics’ understanding, as well as the key work of the Disability Resource Centre in advocating for students negotiating adjustments. Importantly, students highlighted that just knowing about the disability services available, and the possibilities for Access Plans was important.
- Material aspects such as exam room/hall configuration, time spent preparing for exams, time allocation for the exam, time extensions, and travel to/from exam locations could also improve or worsen students’ experiences.
Dissemination
We have published two blog posts relating to the project via the CRADLE blog, which enjoys an international readership:
Upcoming activities
Participatory workshops, including academics, academic developers, and students across Deakin and CQUniversity, will be held in February–May 2021.
We are in the process of recruiting a student member to the Advisory Group, which currently comprises: academics Prof. Denise Woods, Prof. Phillip Dawson, Dr Susan Grimes, Mr Matt Brett; and disability practitioners (and managers of the service at each institution) Merrin McCracken and Cate Rooney. The Advisory Group will help us think through some of the dilemmas we encounter, and comment on the resources and materials which come out of the project.
When we submitted this project, we also had thought students might be returning to campus in the second half of 2020 to undertake more ‘normal’ exams. We thus have much more data about the online exam experience than we anticipated collecting; however, given online exams are likely to continue into the future, we can refocus some of our findings to deal specifically with the online setting in relation to students with disabilities, from low SES backgrounds, or from rural, regional and remote areas.
We look forward to completing the final report later in 2021, which will be published on the NCSEHE website.
Related