In Hungary it's a pipeline. You go in a room, you pick a piece of paper out of a hat, which gives you some task or topic to prepare for. Then you sit down at a desk (the room is filled with about 15 students working already), then you solve the problem or sketch out your notes to present the topic you picked for like 20 minutes, then you go to the professor and you have to, in low voice, explain your solution, the professor asks back and pokes at it to make sure you understand. The professor then gives a grade immediately and you go home.
In Germany, it's usually no random pick of a topic, you just go through the entire semester's material over 30-45 minutes and the professor stops to spend more or less time on each topic to explore your knowledge. You can explain things by making sketches on paper. A PhD student sits there quietly taking notes of what was asked and what the student answers approximately. You leave the room, the prof and the PhD student discuss the performance, decide the grade and you come back, and the professor explains the grade and what you did well and not well.
Since each course needs its eval, assessment is a function of N, not N*K or K. Much easier to scale if there is ~5 evals per student per year than ~20.
I was the PhD student in the room in dozens of these exams, and I toke some of them myself. But that was for classes with a bit less than 40 to maybe 60-80 students. The load for the larger ones was shared among professors, if I remember correctly. But it was certainly the case that a single professor spent 1-2 days doing nothing else but these exams.
And it was 1 professor and 1 PhD students, and those don't count. They essentially grow on trees and are always available, if you need some.
I think education is something better scaled horizontally than vertically. More schools, with more professors instead increasing the student per professor throughput. The classes where I learned the absolute most, and had the best experience, were closer in size to 25 than 100.
The horizontal capacity is ineffectively used by having many degree mills and low-value degrees, but if we re-tooled the knowledge sets we could raise the bar.
As for the quality, I think small classrooms with good teachers probably produces a better outcome than large classrooms with great teachers.
With an oral exam, the prof blocks off 3 days or so, with ~8 students per day and it's done. It's not a huge additional effort compared to all the effort of giving the course over the full semester anyway.
An added, though minor benefit is that the professor might decide to offer a student assistant job to someone who performs well, because it's much easier to judge who is actually good in an oral exam vs in written. There's a reason that job interviews aren't just tests filled out on paper but involve personal interactions.
They are likely prone to more biases than written exams, and the variability among examiners is likely high. My experience is both in taking these exams myself, and also sitting in on them as the note taker many times. Though that was only with one specific professor, and almost always on the same course. But that was in my experience a very fair exam, even for those students that were very noticeably nervous.
The appeal of digital systems, at a social level, is that they can transmit information accurately for almost no marginal cost. They come at the risk of fraud.
With AI the levels of risk have increased. Anyone can now be a fraudster, and it gets harder and harder to prove anything online. Either of us two, in this conversation, could be a well tuned LLM.
So society, collectively and at an individual level, is forced to reassess reliance on purely digital information. Which means that the advances in the technology, actually end up reducing the utility of digital systems more generally.
You can argue that this makes education costly and less accessible. It does. It absolutely does. There is also no alternative except to see higher education get wiped out, which would have devastating economic and social effects.
So we are going back to blue books and vivas.
You don't need to do everything orally. A return to handwritten final exams done without electronic aids would help too. They can be open note as mine were in the mid 70s or even open book.
As for scalability, I really don't see the problem; if we are producing so many qualified people then we surely can use some of them to examine the next generation. It sounds like the argument that often appears on the web where someone, usually an American, claims that a solution that works in, say the UK, can't work in their country simply because their population is larger.
Also, we really should consider if shoe horning every young person into an academic institution is useful either to society or to the person concerned. It certainly has not been an unmixed success in the UK.
And that's before we get to technological changes making that sort of knowledge and those skills much less valuable.
As an aside, accessibility cuts both ways. As a person with ADHD I'd much rather do oral and practical exams.
So we’re probably going to see a split- mass production schools that still act as job training for Industrial Revolution era jobs will keep doing what they are doing. Schools that want to keep their academic prestige will implement these inefficient safeguards and raise prices accordingly. Employers will distinguish between these two types of diplomas when there are enough options out there that are combatting cheating- much like in-person schools are generally valued higher than online educations, or how certain schools (MIT, Stanford) are valued higher then others. This might lead us to the transformation of higher education many have been anticipating, because the current system cannot remain as-is. We all thought technology was the thing causing the upheaval but AI has really accelerated things.
Interestingly, you dont consider your 'inverse' point: that now the manual part of teaching the stuff can (and definitely will) be automated with LLMs.
It will become similar to coding: humans evaluate, LLMs just provide what has been mechanical for a couple hundred years
It can also be graded blindly. That allows for removing–or at least mitigating–biases in evaluation.
Universities in the US charge fortunes and pay instructors a pittance while plowing ever more money into “administrators”. Fire half the administrators and hire more instructors to both proctor exams and give more personalized instruction.
Universities are entirely elective, why would they need to cater specifically to disabilities? Driving schools don't make exceptions for the blind. There are skills that people expect you to have with a certain kind of diploma, and it reflects poorly on the school if they don't have a standard to hold people to. Not being able to present your work in a basic way is one of those mandatory skills that imo makes you unqualified to graduate if you don't grasp it.
I'm not saying you can't make an exception for a person or two who is clearly as good or better than the rest otherwise, but people will always try to misuse as many exceptions as they can, en masse, as an excuse to just avoid doing required work.
It doesn't feel good, but at some point we have to ask, can we support every single case, at the expense of the whole? I'd like to be convinced that's not a necessary tradeoff
The list of conditions that qualify for what will most likely have to be redone completely from scratch.
I'm not joking: you could have an AI quiz the students and test their understanding. That way the tests go deeper into testing the understanding by following through on the answers, and are standardized and fair for everyone. I don't think we are there yet at all but I actually think its not insane for the future. I expect downvotes though/I don't think our generation would agree with this.
What's old is new :)