Using data to help adapt to challenging times

Principal Fellow, Dr Camille Kandiko Howson, opened our 2021 Surveys & Insights Conference: Adapting to Challenging Times on 28 April with a keynote exploring student voice, asking whether it is a formative or summative measure of quality. 

Dr Kandiko Howson is an Associate Professor of Education in the Centre for Higher Education Research & Scholarship (CHERS) at Imperial College London. She is an international expert in higher education research with a focus on student engagement, student outcomes and learning gain, quality, performance and accountability and gender and prestige in academic work. 

In the middle of a pandemic, the student voice has become a political hot potato in England. Camille explained how the Department for Education (DfE) expressed concerns around the primary vehicle for student feedback, the National Student Survey (NSS), but that a review by the Office for Students (OfS) concluded everything was “fine”. This year, participation is optional but most HE institutions have signed up. 

She feels the OfS review was a “missed opportunity” to explore other options. Although one of the primary findings of the survey is that students “love higher education” with consistently high scores around teaching and learning and overall satisfaction, there is a tension between the DfE and students’ opinions as an arbiter of quality. The issue lies with students not agreeing with the government’s view of ‘quality’ courses. Courses which the government views as low quality or value are often rated highly by students, and vice versa. 

However, there are some effective alternative approaches to the NSS, according to Camille.  

Formative approaches 

Advance HE’s UK Engagement Survey (UKES) is about student self-formation, development of societies, human capital and an active and informed citizenry. Camille was involved in the cognitive testing for the survey and reported that students said that even just by reading the questions, it prompted them to think about how they were spending their time and to make changes. 

Ongoing student feedback such as mid-module, student voice and engagement apps, for example, allows students to see changes quickly. 

Camille sees a potential for “voiceless student voice” moving forward: data science, learning analytics and AI. She says it’s a more precise way of capturing what students are doing and there has been a lot more interest in this with the move to online learning this year and the greater use of online platforms and technology. 

 

However, it’s “difficult to gather at an institutional level, let alone comparing cross-sector…Student surveys are one step removed from how students are feeling about their experience, and data analytics are one step removed even further”, she says. 

So this is likely still a way off as a replacement for student surveys and it is not a standardised approach, so what is the future? Camille predicts a greater focus on linking data, better capture of educational/learning gains and support for student mental health and wellbeing. She says student outcomes and employability is an area that’s “very interesting moving forward as we need to adapt approaches from the past due to new economic models and the labour market moving forwards.” Student perceptions of value for money has generated lots of media interest so we may see “more legislation on this in future given the move to online for some courses and institutions”.   

Summative approaches 

Government use of student voice has been as a measure of what works or does not work. Camille says student experience and voice research shows how to help students improve in their own educational experience and the choices they make.  

Currently people are using large data sets and machine learning approaches but often produce obvious insights such as ‘students who attend more lectures get better grades’ so researching student outcomes is a key area. It’s interesting, Camille says, to see which courses are closing – often those which are closed are those which don’t feed directly into clear employment routes, rather than looking at the broad data on student outcomes. A narrow view feeds into the government’s view on tightening access and decreased funding for arts courses. 

A note of caution, however, on ethics and data. Students haven’t had a huge amount of say over what’s done with their data in the past but Camille says this will change as we move to using more data analytics. Learning analytics and AI are in their infancy so she asserts that we "need to work in partnership with students in the future and make students owners of their own data". 

Camille summarised with how the policy sector views quality versus how students view quality. She sees the need for more ongoing feedback options, moving beyond satisfaction so it will be interesting to see how this key final question will be replaced in the NSS. “I would hate to see a question on ‘did you get value for money from your HE experience.’” She argues it is more of a meta-construct, and there are variations in how ‘value’ understood and defined. 

For students to have a high quality experience their feedback must be listened to and valued. She says it’s important that it is acted upon and that “the feedback loop is not just closed but is ongoing”. 

Camille ended with a call to action for analysts of student voice data. “Engage students”- there must be an ethos of “working WITH not FOR students”. 

Maddie is Research and Insights Executive at Advance HE, working across PRES, PTES and UKES. 

Find out more about our student surveys.