On 24 May, Advance HE and Higher Education Policy Institute (HEPI) hosted a breakfast seminar in Parliament considering how best to assess the value of HE institutions. Delegates at the event, Measuring the wrong things?: How should we judge the success of universities in creating value?, debated what alternative measures of value could be used in place of simple-and-flawed indicators like graduate earnings, and whether the sector needed a basket of metrics and other indicators. After the seminar, Advance HE spoke to a number of governors from a range of institutions to gather their views on some of the issues raised.
Higher Education Intuitions are in an era of intense monitoring of student outcomes. Numerical baselines applying to continuation, completion and progression to professional jobs or further study became part of the regulatory framework in May. The Office for Students has already announced that an initial eight institutions will be investigated because of “poor quality courses”, with more to follow.
From a governance perspective, the move has resulted in more scrutiny of student retention rates and employability than ever before, with these areas shooting up the risk register. Deep dives into particular subject areas are being launched across institutions.
News of reviews of provision at a number of institutions and the threat of course closures and redundancies have hit the headlines. The University and College Union has accused the government of trying to get rid of courses with employment records which mean graduate loans are less likely to be paid back.
In the midst of this pressure, governors acknowledge that while student outcomes are vitally important, they raise fears that their narrowness of focus and inability to reveal the bigger picture is already having serious implications across the sector.
Taking creative courses as an example, one governor points out that they might be at the bottom of earnings and employability tables, but the size and strength of the UK creative industries depends on the pipeline of talent they provide.
There is also no clear picture of the extent to which the metrics used by the OfS take account of portfolio careers, freelancing, limit company arrangements and short term contacts – all increasingly common in the gig economy.
For another governor at a post-1992 London university, the question of what students value and the reasons behind the decisions they make is crucial and often ignored in the overconcentration on data.
“Some degrees outcomes are less straightforward in terms of a linear pathway into graduate jobs,” she said. “Students understand this but want to study their subject. It is not a reason to constantly barrack art and design and the humanities. We’ve always had the auntie saying about the history degree ‘what job are you going to do with that?’ But the degree is giving you so much else – it’s giving you the power of thinking, critiquing, reflecting. And the government focusing on degree outcomes as the sole factor in employability is a flawed argument.”
The wider context, often involving factors beyond universities’ control, cannot and should not be ignored in assessments of value, according to governors.
“There is a reliance on narrow economic measures and a lack of appreciation that the UK is a very unequal country,” said a governor at a post-92 university in the north of England. “The UK is an outlier in terms of economic disparities and that will be reflected in student destinations and earnings post graduation.”
One governor raises the issue of lower employment rates among black graduates as a contextual factor impacting some universities more than others.
“If you are black we know that you are disadvantaged in terms of getting that first job,” she said. “There are barriers that get in the way, particularly for universities like us that have focused on widening participation. If you take black students graduating from our physiotherapy and nursing degrees, their employment rates are lower than at a nearby institution which has a reputational advantage, based on prejudice, perception, snobbery and all those things. Graduate outcomes might be put at the door of the university but there’s much more going on. Using graduate employment and the ability to pay back loans as a judge of quality is very reductionist and oversimplified.”
The risk, if context is not taken into account in measures of value, is that certain groups of students – those more likely to drop out, fail to complete or are less likely to get a good job – become a less attractive proposition.
“If widening participation (WP) students threaten your metrics, that is going to become a problem for some universities,” said one governor. “WP is central to our mission here but others will look at the bottom line.”
The governor at a university in the West Midlands argued that the government’s definition of “a good job” was “who gets the highest salary and who gets into the city”.
She added: “There is no one-size-fits-all; what I might prioritise and value, might be different to what you prioritise and value. The university I’m a governor at is very different to the ones in London, for instance, and the aims of many of our students are different. I understand the need for accountability but to try and set numerical thresholds based on what you assume student value to be is just not right.”
In Wales, the concentration on graduate employment metrics is less acute. According to one governor of a new university, Widening Participation is a particular feature of the First Minister’s statements and one of the motivators behind a new bill going through the Senate to bring funding and regulation of all post-16 teaching HE, FE and sixth form colleges under a single regulator.
“The idea is to improve pathways and encourage collaboration, with the underlying aspiration around WP” he said. “From here, the OfS looks very metric driven in comparison.”
Benefits to the individual over and above academic achievement, such as opportunities to broaden horizons, create networks, enhance social skills and boost confidence, are also part of the appeal of a university education, though harder to measure.
“Our students really do appreciate and thrive on that but can those things be captured in a basket of measures?” asked one governor.
Participating in student societies, social enterprises and volunteering provides young people with a range of opportunities, according to one former SU president and student governor from a university in Scotland. Not only are such activities in the student’s own interests, they bring societal benefits.
“It should be possible to measure the number of students that are volunteering, for instance. If you are measuring other things why not measure that?” he asked. “Part of the National Student Survey (NSS) and getting the student experience right is the stuff outside a degree. It is so important to get a sense of community and enrichment, both at the time and later on when you are starting your career.”
These wider benefits of higher education are acknowledged by the government in its Levelling Up White Paper, which refers to universities as anchor institutions and expects them to do more to boost local economies and communities.
But as one governor in the North of England points out, this ambition can run counter to employment outcomes.
“Many students come from and remain in their local areas and that is a good thing in terms of provision of a skilled work force but if local employment opportunities are not here, it will impact on universities’ graduate employment rates,” he points out.
Contributions to local and regional economies can be estimated and bodies such as Universities UK have produced research showing financial values, as have individual institutions, providing important evidence to ministers and the Treasury.
Value added measures, where the starting point of students is taken into account in the assessment of progress, have been used by a number of universities as part of tackling the ethnicity degree awarding gap, and have been mooted as a possible fairer way to measure HE impact more generally. However, valid cross sector measures are hard to come by without some form of one-size-fits-all standardised test that would be anathema to a UK system that prides itself on its diversity of provision.
Human interest stories of the transformative nature of higher education and/or research plotting graduates journeys over time can illustrate the power of HE and could have more impact and cut through with the public and ministers. They could also help counter some current depictions of the sector as riddled with “low value” courses and the view that university is “not worth it”.
“It is the minority view that it’s not worth going to university but there is a danger that this becomes the narrative,” said one governor. “I was the first in my family to go to university and it was big deal then and it still is. We should not be defensive about it.”
From a governance point of view, the numerical threshold regime is top of agendas: one governor described it as creating “a bit of a panic mode”.
“My board of governors has gone straight to the academic board and said ‘what courses are affected’ and it was one of those instances where the academic board were as concerned as the governing body,” she said. “We’ve commissioned some analysis on this to be presented at our next meeting.”
At the Advance HE/HEPI seminar, Susan Lapworth, interim chief executive of the OfS, gave an insight into the regulator’s approach to the new indicators.
On continuation, for instance, she said universities need be on top of the figures and be prepared to explain what they are doing about problem areas. On employment, universities need to “be aware of weaker outcomes, look at what is driving them and consider whether the steps that they are taking are credible and will lead to improvement”.
Governors’ need to provide oversight and challenges to performance, according to the regulator, playing a central role in strategic direction, reviewing decisions and gauging whether the university measures up to its own aspirations.
“I think there’s an assumption that these baselines will only impact certain courses at certain universities,” said one governor. “But I think every institution will be affected to some degree and I think the sector needs to come to terms with that.”
Keep up to date – sign up to Advance HE communications
Our monthly newsletter contains the latest news from Advance HE, updates from around the sector, links to articles sharing knowledge and best practice and information on our services and upcoming events. Don't miss out, sign up to our newsletter now.