Professor Janice Kay, Provost and Senior Deputy Vice-Chancellor at the University of Exeter and chair of the subject-level TEF pilot spoke at Government Events’ The Future of Teaching Excellence Framework in 2019 and Beyond conference. She gave insight into the findings from the TEF subject-level consultation and explored lessons learned from the 2017-18 subject pilot.
In 2018 two subject-level TEF models (A and B) were tested to determine how well they could generate ratings. This included assessing whether the detailed design and delivery aspects were fit for purpose. The Government consultation, which concluded last year, also sought views on these two models.
The A and B models were used to interrogate questions such as: how fair is it for the subject to inherit the rating of the provider and can ratings for subjects be aggregated to indicate an overall provider rating? The findings demonstrated that neither model A or B was particularly satisfactory. Further ‘what works’ experimentation is therefore underway in a second pilot, engaging with around 45 providers and 150 pilot panel members to test a number of refinements and issues. The original three aspects of teaching quality, learning environment and student outcomes & learning gain remain, consisting of the now 11 criteria making up teaching excellence and designed to improve student voice in the exercise. Types of evidence for examining subjects will differ from provider level, for example, and could include explanations of pedagogical approaches in particular disciplines and enhancement activities.
Janice Kay commented,
“An important point to make around provider and subject TEF is that it is an evolutionary process. This is shown in the way TEF panels actually operate. They take a solutions-focused iterative approach to assessment, and the testing of outcomes is built into the operations of panels themselves. There is constant dialogue and feedback with pilot providers and iterative feedback from panellists.”
Last year, post-pilot workshops were held with pilot providers, panel chairs and deputies, which served to interrogate what worked from different perspectives and ask what meaningful changes could be made. The same approach will be taken this year.
Janice shared her thoughts on student engagement with subject-level TEF, saying,
“There is an important question for the TEF and its potential future impact, which is also one of the terms of reference for the Independent Review.
Is subject-level TEF about a meaningful choice for students to help them decide which institution best suits them? Is it about providing accessible information, is it about driving enhancement or all of these things?
It certainly should be about facilitating learner excellence and giving institutions the opportunity to enhance learning excellence. Metrics are not always perfect and proxy measurements are merely a place to start.”
Provider submissions, read and debated by skilled panellists, make the TEF a process that is fundamentally a peer review.
Recent research commissioned by the DfE has shown that after the first year of TEF results 32% of students said they were aware and had some knowledge of the TEF at their UCAS application stage, and a third of those who were aware of the TEF used it as part of their decision making.
There is an interesting and deep question of whether a self-reflective, metrics informed process like TEF can help higher education organisations to enhance their provision. The involvement of students in the process can serve to make it a powerful mechanism in which providers can fully understand the make-up of their student body. Focus on the structural relationships between institutions and their departments can help to drive interventions that can have a real impact.