Hacker News

Hi, former student with social phobia and terrified of public speaking.

Today: adult with no problems doing public speaking in front of hundreds.

What fixed me: more public speaking and not having the option of avoiding the task.

I have always disliked public speaking and oral defense since I was a child, to the point of fairly profound anxiety. But please withhold you woe—these people are, as I am today, adults. Masters students. Coddling them and holding low expectations has proven to be a sort of “killing them with kindness” approach to handling modern youth. People need to be encouraged (and required) to do things they don’t like to become proper functioning adults. I didnt like it, and I still don’t, as it the case with most everyone. But we all grow from it.

Clarification for non-Danes: this post is not entirely accurate. Not all Master's courses in Denmark have oral examinations, and many courses before the Master's level (including elementary school) have oral exams. It is unrelated to the academic level, and instead related to the academic discipline and the size of the course. The latter is in my experience the main constraint, as oral exams are difficult to scale. They usually take 20-30 minutes per student, and if you have a course with 200 students, well, you're in for a long time.

The oral exam format described is also somewhat unusual, in my experience. It is usually the case that there is one internal examiner and one external examiner in the room. More than that would be ruinously expensive. The format varies; usually it's a combination of a short presentation and questions asked by the examiner. Drawing a random topic is very common.

Source: I am a university teacher in Denmark who teaches master-level courses without oral exams, and I have been an examiner for many oral exams at university level.

64 year old nerd with social phobias here. When I was in school, I had to practice oral knowledge tests since grade one. I had my first oral exam in grade six (math) and 2-3 oral exams every year since then. If I hadn't had that experience, I would never have gotten into university. I still don't like verbal communications with people (especially in a tense situations like exam), I'm slow etc, but it's manageable because I had to practice it a lot. I don't think I would have survived in modern "the environment must be made suitable for everyone" school/university.

> Of course, woe betide those with social phobias...

The professors at my grad school had a very amazing tact to get through this as well. They had enough time for specific students who they knew struggled with such issues during the mid semester orals and then had an amazing way of getting answers from them even in the discomfort.

It is amazing how much can be done with soft talking, slow progression and pauses in between from professors who care.

>>> Of course, woe betide those with social phobias....

True but maybe a blessing in disguise. As a shy college kid once (although not with any real disability), being forced to get up in front of people helped me get over it. And I was 10x better after a semester of teaching during a break in my career.

So, I grew up in the former Soviet Union. Starting with ninth grade entrance entrance and exit exams were like that. I remember I had to take university physics exam just to be able to qualify for a consideration. It was fairly rough, but I think in a way, good system because it sifted out the wannabes early on.

> We ask Master's students to give US instruction on a topic (drawn randomly on a card in front of them, from a list they know a few days in advance)

This is also an absolutely great way to ensure the participants really understand what they’re saying, even aside from ai-related concerns. You could always pull definitions straight from textbooks, and write something which mostly makes sense but that you don’t really understand. But you really can’t teach something unless you do understand it, imo. The best command I ever got on a topic was due to TA’ing

This is not general to all subjects in Danish universities. I studied physics in Denmark and while the majority of my exams after the second year match this description (though not all), my impression is that this is not at all the norm. Students in the humanities in my dorm, for example, seemed to have an overwhelming majority of several-day take-home written exams (with no associated oral defense).

I expect a lot of the social anxiety can be mitigated by arranging things so that students get a chance to know the examiners and build a rapport with them ahead of time. (E.g., if they’re an instructor then make sure they have office hours and encourage students to go.) I’ve got my mental health challenges but that’s how things worked in my grad school and I got by just fine.

Fairly normal, in my experience, for Masters with a thesis in the US as well. And I have not uncommon group project presentations undergrad as well--in addition to grad school.

If you want to be an adult functioning in society, being able to briefly talk about your ideas with strangers is important.

That's how my school did it even for high school courses. End of the year exams were infront of the committee. Same thing, you pull a ticket and have few minutes to prepare the answer. I aced physics course, that was a proud moment but credits didnt transfer when we moved overseas...

This is also the case for many graduate programs in the US as well, but what this article is suggesting is that there are now specific oral exams to defend a student's paper, not just general oral exams.

That’s not entirely true. It varies from master’s programme to master’s programme even inside the same Danish university. Assessment is defined by the corresponding programme board.

Thesis/dissertations do have to be defended, but for course assessment that is not a given.

(I run a course and teach as well on other Master programmes.)

This is truly awesome and fantastic, and I hope it becomes a thing all around the world.

that's interesting, i do something similar when helping my 16 year old with his homework. I ask him to teach me the topic first (it's been a while since i took physics and calc hah). By the time he's able to answer my questions so that i can help him then he's answered his own questions pretty much.

I do this at work too. When doing code review i actually do mostly listening and then some questions. If the developer can teach me the code and i understand what they're doing and why then it's usually pretty good code already.

>Of course, woe betide those with social phobias

I can see this being a problem in the US when it comes to "accommodations" in general. Students with an ADHD diagnosis can claim it causes just about any difficulty and get accommodations for it. The local (top ranked, but everyone's seen all the stories coming out of the Ivies...) college here gives students with ADHD double time on exams and has them take it in a special disabilities proctoring center where it's a struggle to get the staff there to even enforce those time limits. Obviously this also makes it infinitely easier to outright cheat as well.

Not a chance you will get past the "my ADHD makes it hard for me to think about these things verbally" challenge at institutions like this.

Welcome to the real world. One cannot be coddled forever.

Best part you can trivially automate this using AI

Oral exams are still common in Germany and many European countries in case of smaller courses taken by 20-30 students in a semester, and sometimes even in larger courses, and the task of examining is then split among multiple professors and lecturers.

In Hungary it's a pipeline. You go in a room, you pick a piece of paper out of a hat, which gives you some task or topic to prepare for. Then you sit down at a desk (the room is filled with about 15 students working already), then you solve the problem or sketch out your notes to present the topic you picked for like 20 minutes, then you go to the professor and you have to, in low voice, explain your solution, the professor asks back and pokes at it to make sure you understand. The professor then gives a grade immediately and you go home.

In Germany, it's usually no random pick of a topic, you just go through the entire semester's material over 30-45 minutes and the professor stops to spend more or less time on each topic to explore your knowledge. You can explain things by making sketches on paper. A PhD student sits there quietly taking notes of what was asked and what the student answers approximately. You leave the room, the prof and the PhD student discuss the performance, decide the grade and you come back, and the professor explains the grade and what you did well and not well.

Oral exams can also compensate a lot for different types of exam anxiety. Blanking on individual points doesn't matter as much, the examiner will help you along or ask along a different path. Doesn't help for someone that completely shuts down, but it's much easier to determine if a student was just stuck on some minor point and actually knows the topic in an oral exam.

They are likely prone to more biases than written exams, and the variability among examiners is likely high. My experience is both in taking these exams myself, and also sitting in on them as the note taker many times. Though that was only with one specific professor, and almost always on the same course. But that was in my experience a very fair exam, even for those students that were very noticeably nervous.

I think you have the right observation but the wrong conclusion.

The appeal of digital systems, at a social level, is that they can transmit information accurately for almost no marginal cost. They come at the risk of fraud.

With AI the levels of risk have increased. Anyone can now be a fraudster, and it gets harder and harder to prove anything online. Either of us two, in this conversation, could be a well tuned LLM.

So society, collectively and at an individual level, is forced to reassess reliance on purely digital information. Which means that the advances in the technology, actually end up reducing the utility of digital systems more generally.

You can argue that this makes education costly and less accessible. It does. It absolutely does. There is also no alternative except to see higher education get wiped out, which would have devastating economic and social effects.

So we are going back to blue books and vivas.

Even in fairly recent times viva voce was common. I had to defend my final year report on a long piece of practical work as part of the final for my BSc in Applied Physics in 1975.

You don't need to do everything orally. A return to handwritten final exams done without electronic aids would help too. They can be open note as mine were in the mid 70s or even open book.

As for scalability, I really don't see the problem; if we are producing so many qualified people then we surely can use some of them to examine the next generation. It sounds like the argument that often appears on the web where someone, usually an American, claims that a solution that works in, say the UK, can't work in their country simply because their population is larger.

Also, we really should consider if shoe horning every young person into an academic institution is useful either to society or to the person concerned. It certainly has not been an unmixed success in the UK.

I'm quite skeptical that industrial-age measures of efficiency are the right way to think about education. Feinman's story about teaching in Brazil comes to mind: https://enlightenedidiot.net/random/feynman-on-brazilian-edu...

And that's before we get to technological changes making that sort of knowledge and those skills much less valuable.

As an aside, accessibility cuts both ways. As a person with ADHD I'd much rather do oral and practical exams.

I graduated in 2000 and remember doing the oral defence. It wasn't a huge deal. You already have a personal relationship with your tutors and they are familiar with you and your communication style. They are not the Gestapo.

This is basically my thought- my school is considering all sorts of things to combat AI cheating, and most of them scale poorly. You can implement them and increase your reputation, but your tuition will have to increase to pay for a lower student/teacher ratio.

So we’re probably going to see a split- mass production schools that still act as job training for Industrial Revolution era jobs will keep doing what they are doing. Schools that want to keep their academic prestige will implement these inefficient safeguards and raise prices accordingly. Employers will distinguish between these two types of diplomas when there are enough options out there that are combatting cheating- much like in-person schools are generally valued higher than online educations, or how certain schools (MIT, Stanford) are valued higher then others. This might lead us to the transformation of higher education many have been anticipating, because the current system cannot remain as-is. We all thought technology was the thing causing the upheaval but AI has really accelerated things.

And what a better time to set optimization aside when schools and universities are getting less and less students each semester due to population decline.

Interestingly, you dont consider your 'inverse' point: that now the manual part of teaching the stuff can (and definitely will) be automated with LLMs.

It will become similar to coding: humans evaluate, LLMs just provide what has been mechanical for a couple hundred years

> A written paper could be handed in and graded without lining everyone up in front of a panel

It can also be graded blindly. That allows for removing–or at least mitigating–biases in evaluation.

It’s a good answer for keeping online education honest. It doesn’t need to be applied to all in-person schools, where teachers distribute spot checks across the course relatively evenly, but it makes perfect sense in conjunction with online testing. You only have to challenge one essay per student per term to catch cheaters with a near-perfect success rate. Assuming a 12 week course with 36 students, that’s qty.3-4 of teacher:student one on one meetings per week for the term; I think that’s doable. If you’ve oversold your teachers and have more than 36 students per, guess you’ll have to increase your labor expenses :D

It shouldn't take long for educators suss out whether students know the subject they are talking about. You don't have to question them exhaustively to determine that they likely did or didn't do the work they are claiming. TFA is about high school projects, not PhD defenses.

Complete bullshit.

Universities in the US charge fortunes and pay instructors a pittance while plowing ever more money into “administrators”. Fire half the administrators and hire more instructors to both proctor exams and give more personalized instruction.

> Students with speech differences, anxiety disorders, hearing impairments, or simply less fluency in the language of instruction face an extra barrier that a written submission does not impose. The written form, for all its flaws, leveled some of that ground. Shall we abandon the differently abled because machines disrupt the academic system and essentially make cheating effective?

Universities are entirely elective, why would they need to cater specifically to disabilities? Driving schools don't make exceptions for the blind. There are skills that people expect you to have with a certain kind of diploma, and it reflects poorly on the school if they don't have a standard to hold people to. Not being able to present your work in a basic way is one of those mandatory skills that imo makes you unqualified to graduate if you don't grasp it.

I'm not saying you can't make an exception for a person or two who is clearly as good or better than the rest otherwise, but people will always try to misuse as many exceptions as they can, en masse, as an excuse to just avoid doing required work.

I don't think the world really works like that. Hearing an elevator pitch has a very different impact from reading a short-form business proposal. And of course the efficiency of the written word is not absolute; in lively discussions and debates, there is no time to put thoughts down into the written word; they had to be uttered. I am personally of the opinion that a successful university graduate should not only be able to write down their ideas in words but also be able to speak to defend their ideas.

"The oral system scales poorly. One examiner can only hear so many students in a day."

I'm not joking: you could have an AI quiz the students and test their understanding. That way the tests go deeper into testing the understanding by following through on the answers, and are standardized and fair for everyone. I don't think we are there yet at all but I actually think its not insane for the future. I expect downvotes though/I don't think our generation would agree with this.

What's old is new :)

But even with the written exam, wouldn't the bottleneck still be there where the professors still need to read through all the papers? I guess its definitely more efficient and asynchronous, but I can't imagine by much assuming the same and equal rigor is applied.

IMO this kind of thing adds work and stress for the best students. If I'd been faced with something like this while I was in school I would have felt the need to basically curate a parallel construction AI discussion, knowing it would be judged. I understand the theory that, in the age of AI, the prompts are more interesting than the outputs, but I'm not convinced this is the way.

Things like the Denmark approach (and/or in-person written responses) seem like a better direction. Treat take-home writing the same as we've treated take-home math questions - as useful practice (for those who choose to actually engage with it), but not a significant input to grading.

You suddenly realize the foremost expert on the research in your dissertation…is you.

Its serial nature also means that an evaluator's standards can drift from students with early exams to those with later exams. Oral exams can introduce issues of sex and ethnic biasing.

Arguably, the problem with higher education is one of volume.

I'm teaching a university math course next semester and I've been messing around with a solution to this that my old analysis professor used to employ. Every week, they take home a handful of theory questions. These aren't turned in for credit. But, every week, there'll be a short quiz where one of those questions are asked verbatim, along with a small follow up.

That felt like a very natural solution to me. Of course, worst case scenario, they just blindly generate and memorize the proofs without thinking. But I think the follow ups will catch them in that case. Moreover, I choose to believe that >90% of my students are willing to at least put in the work of scrutinizing those proofs which is honestly not bad exercise in its own right.

I would love to do oral exams directly, the logistics with the number of students I have just make that hard. I know we're trialing it with the less populated special topics courses.

Only as an honor system for students who want to learn the material.

But worthless for assessments and assigning grades.

Only the dumb ones are obvious. In my school (SFU/BC) most of the students in almost all of my classes cheated. VERY FEW got caught.

I heard instructors say all too often that it was obvious when somebody was cheating, but somehow they missed the guy paying his cousin to write the final.

Thos whole debate is happening because people think a degree matters. Because getting a piece of paper is more important than knowledge. Students will continue to cheat (AI or not) as long as the goal is to get a stamp.

Did you know that there is a foolproof way to cheat on any exam? You can just store the answers in your brain. They won't even notice you cheated.

Of course they could, in fact it'll probably be great prep if they read it and understand it instead of just memorising the answer.

If they just memorise the answer any smart teacher can trip them up by just rephrasing things or developing tangents to see understanding.

In school before exams, I'd often make little cheat sheets about things that you'd need to memorise like formulas for areas and volumes or certain physics equations (we weren't allowed any reference material for that on exams). The very act of writing them all down before the exam would often boost my memory enough to the point I'd just recall them.

At this point you're just describing learning using buzz words.

I'm also surprised. In Portugal primary, middle and high school tests and exams are all proctored and the vast majority of them written by hand.

You've touched on something real here

More like a tired trope.

At some point they will have to be won't they? Too many students.

At scale, I imagine something similar to the Baseline Test in Bladerunner 2049.

In my part of Europe oral exams have been a staple even in highschool and across unis. I was pretty surprised when I've seen that other parts of the world don't complement written testing with oral understanding checks.

I hope at least the homework is good prep for the exams, otherwise if its grade value is marginal there's little point from the PoV of a student that just needs to pass the class to take it seriously

I'd never pay to have you as a teacher, I'm sorry. Maybe your move makes a sense for the reputation of your university's degree, but as a student willing to pay to learn remotely, I'd not accept to be treated like children.

> In my opinion, the bulk of traditional education has been a mix of memorization of facts, knowing inference rules, and applying inference rules to those facts, coupled with recall.

These are important skills to develop, even in a world with LLMs.

> All this "reject AI, do it by hand" is just insanity at worst and laziness at best.

If a 3rd grader complained about having to learn multiplication because calculators exist, would you agree with them?

Why do we make kids learn how to do math if they can pull out their phone and open the calculator app? Because learning how to learn is important and there is value in understanding how the answer is produced, even if you have a machine that can produce it for you.

Universities can teach a separate class on how to use LLMs. The existing classes should be focused on teaching their existing subject matter, not becoming an extension of a how-to-LLM class

Depends on what skills you want the students to have.

In e.g., philosophy, allowing students to write their assessments with the help of LLMs changes the students’ depth of understanding, as well as their ability to synthesize, formalize, and drum up their own arguments and objections.

These are all useful skills, even in a post-LLM world.

Two points:

There are two kinds of engineers in Finland. Some received a more theoretical education at research universities. Most went to more applied institutions that focus more on practical skills. Regardless of what employers say, they generally prefer the theoretical engineers from research universities. Because the higher status of those universities attracts more talented individuals, because theoretical understanding tends to stay valid longer than practical skills, and because it's easier to learn practical skills at work.

The kids who start their studies today are supposed to graduate in 2030, and they will probably retire around 2080. Focusing too much on the skills their early employers might want in the 2030s would be a huge misallocation of resources.

Would it work in Wisconsin, Minnesota, South Carolina, Alabama, Louisiana, Kentucky, Oregon, Oklahoma, Connecticut, Utah, Nevada, Iowa, Puerto Rico, Arkensas, Kansas, Mississippi, New Mexico, Idaho, Nebraska, West Virginia, Hawaii, New Hampshire, Maine, Montana, Rhode Island, Delaware, South Dakota, North Dakota, Alaska, DC, Vermont or Wyoming?

They all have a lower population than Denmark.

All but six states (CA, TX, FL, NY, PA, IL) have less than double the population of Denmark.

Top US universities actually have higher teacher/student ratios than top EU schools, and I wouldn't be surprised if this pattern holds in general, so it would be easier to do in the US than it is here.

Piling on. Why? Are class sizes that different? US and China have many more professors too, is it not proportional?

The US consists of many small parts too, and it may work in each of those small parts.

The important variable is the staff/student ratio, not the size of the country. With few students per class, or many teachers, this scales fine. I'm running similar exams for my (UK) Masters modules, but colleagues are reluctant to do it for the enormous compulsory survey courses as there are several hundred students in them (vs up to about 50 in the MA ones).

Why not? This is such a thought terminating statement. No actual reason.

Can you explain why? As I understand it, you're supposed to be able to talk about the work in which you already invested the mental energy to write. Writing involves much more "brain work" than speaking, so if you've truly written the work in question, you should be capable of speaking about it -- unless you have some sort of disability, in which case, there are probably alternative avenues.

This is definitely something you can't force onto students on a whim. This is a something that can be taught and worked up to though.

Part of college is growing as a human, and part of that is social interaction.

For the same reason a lot of universities in Europe require you to take philosophy. It's completely unrelated to C++ or data-structures, but it adds to your acumen as a whole.

Not that different from some universities that require you to be able to swim[0].

[0]: https://perec.columbia.edu/swim-test-requirement

Yeah as much as I like the idea, I know there will be some brilliant and driven students who would have a hard time with this due to temperament. For me, if I’m excited about something, I can talk for hours. Maybe not so for everyone though. Then again, not all brilliant and driven students test well either, so… pick your poison?

If you don't have enough of a mastery over a subject to rapidly respond, maybe you are not fit to pass exam certifying your mastery over a subject.