The OfS meets AI
England’s higher education regulator, the Office for Students (OfS), now reports that lecturers find coursework “close to impossible” to assess because of AI, and it wants universities to return to traditional exams. This reads, to me, at once well-intentioned and self-serving. Yes, students offloading thinking and judgment to AI is a serious issue, but I do not think the Luddism of a return to handwritten exams and vivas is the solution. The assessment formats it recommends are what it is currently equipped to audit, but what might be worth learning in the age of AI increasingly feels divorced from such assessments.
The higher-level issue here is that the OfS, a centralising authority, is struggling to deal with AI, a far stronger decentralising force than the internet was for education, as it allows highly scalable personalised feedback. So, the two are currently struggling to coexist.
But the university — students and teachers — is caught in their crossfire; the OfS can sanction it, but not adapting to how AI potentially alters the economic landscape could also affect student recruitment. Although the report recognises other assessment approaches, I worry that the threat of sanctions will make traditional exams the default: universities may see them as easier to defend to the regulator, even when another approach would better serve learning.
The nature of standardisation and centralisation of education in England’s university system can come as a surprise to someone educated in America (like me, from the University of California). There, instructors have more freedom in what they teach, how they teach, and how they assess across the semester. This decentralised flexibility also results in newer courses or updates to existing materials that keep up with sudden progress. In England, I have heard (from others who have been here longer than I) that there are limits on updating course materials; the comparative administrative burden also feels higher, which means that academics are under-resourced and also not incentivised to update teaching materials. Scott Aaronson’s new AI alignment seminar shows one route for introducing emerging topics. I have not found an equally clear route in my own teaching environment. That does not mean none exists, but I would expect such a route to be visible, with academics actively encouraged to use it.
This adaptability matters when a transformative new technology like AI appears that changes both what students can do and what they need to learn.
Doing a good job of bringing AI into education needs decentralised experimentation. For this, every teacher ought to be incentivised for augmenting their pedagogic approach with AI; this could happen in the classroom, in new course materials, and potentially even in new assessment formats. This is a lot of work with scant incentives, and penalties such as sanctions are unlikely to help. AI can genuinely help each student learn, but teachers are probably the ones who need to learn to use it better first, and then teach it to students. Both need a reason to try.
Some reasons are plain: most students come to university to be better equipped for entering the workforce — as job seekers, job creators, independent creatives or artists — by the end of it. So we must align their learning towards that. What the OfS is suggesting seems to serve exams as an end, instead of seeing education itself as a means.
Handwritten in-person exams are increasingly ineffective for the world students graduate into. It is quite clear that in an AI-first world, humans will not rely on an ability to handwrite essays, reports, code and maths. Instead, they will steer AI agents, then review, verify and edit the AI-generated work (or re-steer the AI to do the task better). In other words, they will audit AI. If anyone should understand the importance of auditing AI-generated work, it should be the OfS, right?
In theory, a viva could work better, but the issue is scale: one or two academics interrogating hundreds of students on a single course is unlikely to give consistent assessment or to probe a student’s deeper conceptual understanding.
In this short essay, I presented kernels of a scalable alternative across a programme. AI-free exams inspired by the Indian Institutes of Technology entrance tests could assess technical reasoning and judgment at scale. Students could work through problems on paper, then select answers or enter numerical results on a computer, much as in the SAT or the GRE. Automatic marking would reduce the workload of grading handwritten scripts; the questions could still demand more than recall. Alongside these AI-free exams would sit a longer project where AI helps students, as a partner, achieve objectives they never could before. This would allow instructors to develop AI skills alongside students, and also to demonstrate the comparatively more mature human judgment that students could learn through osmosis. As that essay proposes, universities could assess students’ reasoning and responses to feedback continuously, rather than relying only on end-of-term exams. I expect this idea to be met with resistance, because it decentralises education while also proposing wholesale programme changes. At present, I am afraid that might be the only way.
The gap between higher education and employers was already quite big, and AI is not going to narrow that gap. If we don’t show students how to work with it better, they will enter the world disempowered. But if we teach them to work with AI better, we might also understand the gaps in their usage of it. In other words, teachers need to learn to use AI better. I suspect the OfS does too.
Given that we are entering the age of decentralised education, and that the United States already shows how this worked in a pre-AI world, I am unsure if the OfS should exist, at least in its current form. I don’t think it can audit this moment with its existing strategies and frameworks. If it tries, it might end up holding students back. Perhaps it is the auditor, and not the technology, that needs to be reviewed.