The Return of the Oral Exam

Welcome to AutomatED: the newsletter on how to teach better with tech.

Each week, I share what I have learned — and am learning — about AI and tech in the university classroom. What works, what doesn't, and why.

In today’s missive, Caleb considers the pros and cons of oral exams. They’re gaining in popularity due to the rise of the remote classroom and LLMs.

New technology may provide a reason to return to old ways.

A recent article from WSJ suggests that more professors are experimenting with oral exams.

Seeing as how LLMs have worsened the plagiarism problem, AI detection tools are of unclear relevance, and AI aces assessments from advanced courses, this is an alternative that calls for exploration.

Moreover, there is some work showing the unique efficacy of oral examination, not to mention hundreds of years of precedent. Oral interviews are a common occurrence in job interviews today. Civil service and academic institutions across multiple continents have implemented oral exams.

This isn’t to suggest that oral exams are the solution for every class in every field. Still, it is enough to warrant more discussion of the benefits and costs.

🗣️ A Beneficial Approach to Learning and Assessment?

For many professions, an oral examination is closer to job assessment and performance. There are, of course, debates about how teachers should weigh employment considerations. Is it more important to provide career value or other educational goods to students? Insofar as this matters for the field and teacher under consideration, it's a reason to favor speaking assessments.

Several papers find that students generally prefer oral exams. For instance, in the case of chemistry, see Lee Roecker’s “Using Oral Examination as a Technique To Assess Student Understanding and Teaching Effectiveness” and David E. Gardner and Andrea N. Giordano’s “The Challenges and Value of Undergraduate Oral Exams in the Physical Chemistry Classroom: A Useful Tool in the Assessment Toolbox.” They find that students experience the tests as engaging and appreciate the link to their professions.

Relevant to LLMs, it’s harder to cheat on oral exams than a written take-home. Of course, this point applies to different exam types, such as in-class exams forbidding electronics too.

There’s evidence that students who take oral exams retain better than those who take written ones. For example, in “Oral Assessments: Improving Retention, Grades, and Understanding”, Mary A. Nelson found that at-risk students who offered voluntary oral assessments performed better in Calculus I and II than those who were not. Roecker reported that students “generally perform at a higher level than on written examinations over the same material.”

Finally, Allison S. Theobold argues for the use of oral exams in statistics classes in “Oral Exams: A More Meaningful Assessment of Students’ Understanding” as they allow a teacher to better gauge how students have (or have not) internalized material. A simple follow up after a mistake can distinguish between a student who simply misspoke and one who completely misunderstands a concept. Written exams do not allow for this opportunity.

😬 An Unnecessary Stress?

Some students undoubtedly find oral exams stressful. Whether that is a significant cost, depends on how much weight a professor puts in decreasing the amount of stress in the classroom. Students report that stress is the largest impediment to academic performance and the level of stress correlates with lower GPAs.

There are a few points to consider here. First, as noted, many students prefer oral exams either because it matches their style, are more engaging, or more closely matches the employment context.

Second, Theobold reports that most students remarked that the exam “wasn’t as bad as I thought it would be” – a remark that fits with my anecdotal impression and understanding of the literature.

Third, oral exams may be less stressful for students that struggle with writing, such as non-native speakers or dyslexics.

Finally, as Helen Hazen notes in “Use of oral examinations to assess student learning in the social sciences” those who find speaking especially stressful are precisely the ones that may find the exercise most valuable. If I’m permitted a therapeutic analogy, the best method for overcoming various paranoias is exposure therapy.

Concerning the cost for professors, oral examinations demand more time. It requires logistical work with a fixed 10- to 20-minute exam time per student. Whether or not this constitutes more work on net is going to depend on the field and implementation of exam options. To ease any logistical headache, Theobold suggests using a tight scheduling system, clear rubric, and mechanism to avoid students walking in on each other’s exams. As one professor reported to me:

🏛️ 🚀 Traditional and Technological

The discussion of costs and benefits is relatively straightforward. Whether speaking exams are best for your class depends on the field, level, and other class details.

What's striking is how new technology may push toward old forms of assessment.

In antiquity, students underwent few, if any, written examinations. Instead they were expected to demonstrate their mastery of rhetoric in performance. Orators like Cicero and Quintillian doubled as moral philosophers and teachers of rhetoric. They took their teaching practice to develop not merely the speaking skills of their students, but their character as well.

While contemporary oral examinations tend to not incorporate moral teaching, they can still mirror the broad perspective that characterized the ancient approach.

By repositioning speaking at the core of education, LLMs have the potential to indirectly foster a sense of educational ambition and breadth of vision.

🔗 Links