NCSEHE research project update — Designing Equitable Principles for Performance Based Funding

You are reading: NCSEHE research project update — Designing Equitable Principles for Performance Based Funding

The NCSEHE conducts an annual Research Grants Program, building a solid evidence base to improve higher education access and outcomes for students from disadvantaged backgrounds.

Designing Equitable Principles for Performance Based Funding, led by Dr Andrew Harvey (La Trobe University) is one of the 13 projects selected in the 2017 funding round.

Overview

This project aims to outline effective design principles of performance based funding (PBF) models, to protect and support student equity in Australian higher education.

Current Government proposals include plans for a mainstream PBF program worth A$500 million, as well as a smaller PBF element of Higher Education Participation and Partnerships Program (HEPPP) funding. These proposals are in addition to recent performance-based measures in Indigenous support funding.

Drawing on evidence from the United States (US) and United Kingdom (UK) and an analysis of national data sets, the project will explore principles required to support identified student equity groups, and to ensure equitable assessment of admissions, student success, and graduate outcomes.

Project activities and preliminary findings

The literature review has revealed that performance funding design in higher education is a complex endeavour, and at each level of the design process, design flaws and unintended consequences can creep in.

Performance funding may induce higher education institutions to change their enrolment patterns to discriminate against students that are a greater risk of reflecting poorly on institutional performance.

Higher education institutions may also inflate their marking and credentialing practices in order to pass and complete more students, thus exacerbating “credentialism” and grade inflation. Both of these unintended consequences can have a disproportionate adverse effect on the academic and graduate success of students from equity groups.

The literature review outlined performance funding schemes implemented in Australia, the UK, and the US.

Australia has implemented three main performance funding schemes over the last 30 years, and all three were abandoned after just several years of operation. Performance funding is currently used in Australia to allocate some research funding, as well as other smaller equity specific grants to “Table A” institutions.

Overseas, by 2016 more than 40 states in the US either had implemented, were developing, or were considering, performance funding policies for their public higher education sectors. This diversity of models provides a rich array of performance funding models to learn from, and analyse.

A small number of studies have sought to evaluate the efficacy of performance funding in achieving aims such as improving completion, and results are mixed. Many of the careful statistical studies find that there is a lack of evidence that performance funding has resulted in improved performance. Some find no immediate effect, but a small positive effect over time after implementation. Other studies find that performance funding has led to fewer enrolments for “at-risk” student groups, and also may lead to more completions in lower level qualifications (institutional under-matching). However, recent research suggests that the use of premium payments for “at-risk” students can potentially improve access for some minoritised groups, providing payments are substantial and sustainable.

The UK is several years into a large performance framework that will eventually be used to determine funding increases for participating higher education institutions. The Teaching Excellence and Student Outcomes Framework (TEF) process develops an initial hypothesis of an institution’s performance based on student satisfaction, completion, and graduate outcomes. Higher education institutions also submit contextual data with metrics split by equity groups and other characteristics, and a more qualitative submission that details current activity the higher education institution is undertaking to improve. It is still too soon to tell how successful the TEF is in effecting improvement, and the UK Government is still adjusting the framework, including developing a learning-gain metric; a teaching intensity metric; subject level assessment; and a grade inflation metric; as well as a longitudinal graduate outcomes measure. Early results, however, suggest the potential for disruption to traditional institutional rankings and a broadening of conceptions of quality.

Thus far, researchers have identified three strands of data analysis to undertake using national data sources.

The literature review identified part-time study, external study, lower ATAR, and other basis of admission, as key risk factors for academic outcomes such as student retention and degree completion. Most of the studies found these factors were better predictors of risk than equity group status. With performance funding, reducing these “riskier” types of admission and enrolment may be incentivised for a higher education institution. The 2016 national student enrolment data was used to analyse the nexus between equity access patterns and these non-equity specific admission and enrolment categories of risk. Most of the official equity groups were overrepresented in these admission and enrolment categories.

The second strand of analysis involves examining stratification of equity groups between higher education qualification levels, fields of education, and institutions. A recent UK paper calculated the socioeconomic segregation between institutions, revealing that some universities are highly segregated. The segregation of students from different backgrounds is undesirable in itself, but also has implications for opportunities to engage in further study. Equity groups are overrepresented at low levels of qualification, and have very low representation at higher qualification levels. Designated Commonwealth Supported Places (CSPs) for postgraduate coursework, and research training places, tend to follow patterns of institutional segregation, to the extent that undergraduate study at a particular higher education institution increases the chances of postgraduate study at that institution. This pipeline effect deprives many underrepresented students of opportunities for postgraduate study. higher education institutions that disproportionally serve underrepresented groups also then have more limited options for improving their graduate outcomes than higher education institutions that receive larger shares of designated postgraduate CSPs and research training funding.

The research team plans to examine the graduate outcomes and student satisfaction of equity groups. It is noted though that students with a disability, and students from non-English speaking backgrounds have poorer than average employment outcomes, and lower levels of satisfaction. There are important questions around the extent to which higher education institutions can be held responsible for student satisfaction and graduate outcomes. With student satisfaction particularly, there are questions around the validity of the instrument. Attaching funding to graduate outcomes may have perverse consequences such as narrowing the discipline mix to more vocational courses. Even if this is an intended policy outcome, predicting labour market demand in three to six years when many students graduate is problematic for governments and higher education institutions.