Measuring success: An innovative approach to evaluating Indigenous higher education
A new report has called for urgent reform in the evaluation of Indigenous higher education policy and practice if student outcomes are to continue improving.
The research, led by Professor James Smith under a 2017 NCSEHE Equity Fellowship, recommends the development of a performance and evaluation strategy, supported by improved collaboration between Indigenous leaders, government and universities.
This report aims to unpack ways to strengthen evaluation in Indigenous higher education contexts in Australia. It is based on the outcomes of a 2017 Equity Fellowship funded by the National Centre for Student Equity in Higher Education (NCSEHE), and hosted through the Office of Pro Vice-Chancellor — Indigenous Leadership at Charles Darwin University (CDU). The Equity Fellowship was undertaken by Professor James Smith, with the support of Kellie Pollard, Kim Robertson and Fiona Shalley. An Expert Project Advisory Group was established to guide the direction of the project from the outset.
The Review of Higher Education Access and Outcomes for Aboriginal and Torres Strait Islander People (Behrendt, Larkin, Griew, & Kelly, 2012) provided a clear mandate for investing in policies and programs that support Indigenous pathways, participation and achievement in higher education in Australia. While there have been notable investments and significant national reforms in Indigenous higher education over the past few years, the recommendation within this report to develop a monitoring and evaluation framework is yet to be actioned. Similarly, in 2015, prior to its abolishment, the Aboriginal and Torres Strait Islander Higher Education Advisory Council (ATSIHEAC) recommended the development of a ‘performance framework’. This has also remained unactioned. As such, there remains minimal publicly available evaluation evidence in this space. In particular, there is scant evaluation evidence about program and policy effectiveness — that is, what does or does not work and why.
Interestingly, a similar trend has been noted in the broader Indigenous affairs landscape in Australia. Concern has consistently been raised about the lack of quality evaluation evidence generated through Commonwealth and philanthropically funded Indigenous affairs programs (Hudson 2016). As such, the Productivity Commission (2013), Australian Government Department of Prime Minister and Cabinet (2017) and the Australian National Audit Office (2017) have all acknowledged the importance of strengthening evaluation in Indigenous program and policy contexts across Australia, including that relating to higher education.
This report brings these two national conversations together. It begins by providing a snapshot of Indigenous higher education participation and achievement in Australia. This provides a background as to why evaluation in this context—which draws on Indigenous perspectives—is important now, more than ever. We then draw on recent academic scholarship and grey literature to discuss what we currently know about evaluation in Indigenous higher education.
The report then presents our findings from a qualitative research study involving 38 individual interviews and one group interview with two participant groups — (a) Indigenous scholars within higher education institutions; and (b) government policymakers with a role in equity and/or Indigenous higher education program and policy development and reform. The study asked questions about the current challenges and opportunities associated with undertaking evaluation in higher education contexts; the enablers and barriers associated with using evaluation evidence in policy and programs contexts; and ways to strengthen evaluation moving forward.