Colleges revive oral exams to combat generative AI as students submit perfect work but can't explain it

Perfect homework, blank stares: Why colleges are turning to oral exams to combat AI

The assignment involves no laptop, no chatbot and no technology of any kind. In fact, there's no pen or paper, either. Instead, students in Chris Schaffer's biomedical engineering class at Cornell University are required to speak directly to an instructor in what he calls an "oral defense." It's a testing method as old as Socrates and making a comeback in the AI age. A growing number of college professors say they are turning to oral exams, and combining a variety of old-fashioned and cutting-edge techniques, to help address a crisis in higher education. "You won't be able to AI your way through an oral exam," says Schaffer, who introduced the oral defense last semester. Educators are no longer naively wondering if students will use generative AI to do their homework for them. A big question now is how to determine what students are actually learning. College instructors across the U.S. are noticing troubling new trends as generative artificial intelligence becomes more sophisticated. Take-home essays and other written assignments are coming back perfect. But when students are asked to explain their work, they can't. The long-term impact of AI use on critical thinking remains to be seen, but educators worry students increasingly see the hard work of thinking as optional. At the University of Pennsylvania, Emily Hammer, an associate professor of Middle Eastern Languages and Cultures, now pairs oral exams with written papers in her seminar classes. "It comes across as if we're trying to prevent cheating," Hammer says. "That's not why we're doing this. We're doing this because students are actually losing skills, losing cognitive capacity and creativity." Hammer forbids AI use on all writing assignments but tells her class she knows she can't enforce that. However, if they haven't written their papers themselves, defending the material face-to-face will likely be "a very stressful situation." Hammer's class is part of "a massive shift toward in-person assessments," both written and oral, at Penn, says Bruce Lenthall, executive director of the school's Center for Teaching and Learning. The Ivy League school is one of a small but growing number of universities that have started running faculty workshops on oral exams. Oral exams are not traditionally part of the modern American undergraduate system, unlike certain European universities. For instance, in the Oxbridge tutorial system in England, students meet faculty for weekly discussions. Some U.S. colleges saw a move toward oral exams during the COVID-19 pandemic to address concerns about online cheating, and interest has intensified since the launch of ChatGPT in 2022. During the pandemic, engineering professor Huihui Qi launched a three-year study at the University of California, San Diego on how to scale oral exams. Several universities have since invited her to provide faculty workshops or discuss her research. At New York University, several types of oral assessments are on the rise. More faculty are requiring office hours, assigning presentations and cold-calling on students in class. Instructors are saying, "I need to look my students in the eye and ask, 'Do you know this material?'" says Clay Shirky, vice provost for AI and technology in education. One NYU professor has put a modern spin on the traditional oral test. Panos Ipeirotis, a professor at NYU's Stern School of Business, unveiled an AI-powered oral exam last semester for the final exam in a class on AI product management. He calls it "fighting fire with fire." Students log in from home, at any time that fits their schedule. A voice cloned from a business school professor greets them. "Hi there," says the voice on their screen. It asks for the student's name and school ID number and then says, "I'm ready to conduct your exam today." The chatbot starts with questions about a final group project and drills into details based on each student's answers. If the student stumbles, the AI agent gives them clues, along with criticism and positive feedback. Ipeirotis grades the exams separately, also with the help of AI. "We wanted to check: Do you know what your team did? Were you a free rider? Did you outsource everything to AI?" says Ipeirotis, who designed the tool with ElevenLabs, a company that develops generative AI voice agents to conduct job interviews. Students in the class this semester are redesigning the AI agent to smooth out some kinks, and Ipeirotis plans to use it in all his future classes. "I want oral exams everywhere now. I want to pair it with every single written assignment," says Ipeirotis. "I don't trust written assignments anymore to be the result of actual thinking." Feedback from students last semester was mixed. Business major Andrea Liu found the chatbot's voice to be surprisingly human, but the conversation felt choppy with odd pauses. It asked multiple questions at once, which was confusing. And it was jarring to hear a voice but not see a person. "It felt kind of awkward to be talking to what was pretty much a blank screen," says Liu, 21. But, she agreed with worried educators: "There is no perfect world where AI exists and kids are not abusing it." Across the humanities and STEM disciplines, like computer science, educators worry that students who skip the mental struggle that is necessary for problem solving won't develop the skills they need to advance in upper-level classes and careers. That's why Schaffer, the Cornell professor, introduced the oral defense in his biomedical engineering class. He requires students to sign up for 20-minute sessions of Socratic-style questioning after submitting written problem sets, which are assigned several times each semester. With a class of 70 students, Schaffer splits the job with his teaching assistants. They no longer grade the written problem sets, just the oral defenses. He calls it "incentivizing" his students to do the work, or at least understand it enough to explain. Schaffer's class is highlighted in a new "Oral Assessment Workshop" offered by Cornell's Center for Teaching Innovation. Other examples at Cornell: a religious studies professor who now holds 30-minute "final conversations" with students instead of a final exam; and another engineering course where the professor gives four-minute mock interviews to each student in a 180-person class. Skeptics point out oral exams can be unsettling for students who are shy or have serious anxiety, but clarifying the format ahead of time and starting with softball questions can help, says Carolyn Aslan, who leads Cornell's oral exam training. "Sometimes it's actually good to get that quiet student one-on-one, and you finally get to hear from them. Sometimes that is the breakthrough," Aslan says. Several of Schaffer's students say they felt nervous at first but ended up preferring the oral exam. "I honestly liked it a lot," says Cornell junior Olivia Piserchia, a biomedical engineering major. She initially found the oral defense nerve-wracking but came to value the one-on-one time with instructors. It kept her from feeling lost in a large class and helped her build the skill of articulating her technical knowledge, as she would need to in a job. "Having that live check-in holds you accountable," says Piserchia. "It's a lot harder to look people in the eyes and say out loud, 'I don't know this.' And, that makes you realize, 'I should study this.'" ___ The Associated Press' education coverage receives financial support from multiple private foundations. AP is solely responsible for all content. Find AP's standards for working with philanthropies, a list of supporters and funded coverage areas at AP.org.

'You won't be able to AI your way through an oral exam': Colleges have an Ancient Greek-style solution to the Gen Z stare | Fortune

The assignment involves no laptop, no chatbot and no technology of any kind. In fact, there's no pen or paper, either. Instead, students in Chris Schaffer's biomedical engineering class at Cornell University are required to speak directly to an instructor in what he calls an "oral defense." It's a testing method as old as Socrates and making a comeback in the AI age. A growing number of college professors say they are turning to oral exams, and combining a variety of old-fashioned and cutting-edge techniques, to help address a crisis in higher education. "You won't be able to AI your way through an oral exam," says Schaffer, who introduced the oral defense last semester. Educators are no longer naively wondering if students will use generative AI to do their homework for them. A big question now is how to determine what students are actually learning. College instructors across the U.S. are noticing troubling new trends as generative artificial intelligence becomes more sophisticated. Take-home essays and other written assignments are coming back perfect. But when students are asked to explain their work, they can't. The long-term impact of AI use on critical thinking remains to be seen, but educators worry students increasingly see the hard work of thinking as optional. At the University of Pennsylvania, Emily Hammer, an associate professor of Middle Eastern Languages and Cultures, now pairs oral exams with written papers in her seminar classes. "It comes across as if we're trying to prevent cheating," Hammer says. "That's not why we're doing this. We're doing this because students are actually losing skills, losing cognitive capacity and creativity." Hammer forbids AI use on all writing assignments but tells her class she knows she can't enforce that. However, if they haven't written their papers themselves, defending the material face-to-face will likely be "a very stressful situation." Hammer's class is part of "a massive shift toward in-person assessments," both written and oral, at Penn, says Bruce Lenthall, executive director of the school's Center for Teaching and Learning. The Ivy League school is one of a small but growing number of universities that have started running faculty workshops on oral exams. Oral exams are not traditionally part of the modern American undergraduate system, unlike certain European universities. For instance, in the Oxbridge tutorial system in England, students meet faculty for weekly discussions. Some U.S. colleges saw a move toward oral exams during the COVID-19 pandemic to address concerns about online cheating, and interest has intensified since the launch of ChatGPT in 2022. During the pandemic, engineering professor Huihui Qi launched a three-year study at the University of California, San Diego on how to scale oral exams. Several universities have since invited her to provide faculty workshops or discuss her research. At New York University, several types of oral assessments are on the rise. More faculty are requiring office hours, assigning presentations and cold-calling on students in class. Instructors are saying, "I need to look my students in the eye and ask, 'Do you know this material?'" says Clay Shirky, vice provost for AI and technology in education. One NYU professor has put a modern spin on the traditional oral test. Panos Ipeirotis, a professor at NYU's Stern School of Business, unveiled an AI-powered oral exam last semester for the final exam in a class on AI product management. He calls it "fighting fire with fire." Students log in from home, at any time that fits their schedule. A voice cloned from a business school professor greets them. "Hi there," says the voice on their screen. It asks for the student's name and school ID number and then says, "I'm ready to conduct your exam today." The chatbot starts with questions about a final group project and drills into details based on each student's answers. If the student stumbles, the AI agent gives them clues, along with criticism and positive feedback. Ipeirotis grades the exams separately, also with the help of AI. "We wanted to check: Do you know what your team did? Were you a free rider? Did you outsource everything to AI?" says Ipeirotis, who designed the tool with ElevenLabs, a company that develops generative AI voice agents to conduct job interviews. Students in the class this semester are redesigning the AI agent to smooth out some kinks, and Ipeirotis plans to use it in all his future classes. "I want oral exams everywhere now. I want to pair it with every single written assignment," says Ipeirotis. "I don't trust written assignments anymore to be the result of actual thinking." Feedback from students last semester was mixed. Business major Andrea Liu found the chatbot's voice to be surprisingly human, but the conversation felt choppy with odd pauses. It asked multiple questions at once, which was confusing. And it was jarring to hear a voice but not see a person. "It felt kind of awkward to be talking to what was pretty much a blank screen," says Liu, 21. But, she agreed with worried educators: "There is no perfect world where AI exists and kids are not abusing it." Across the humanities and STEM disciplines, like computer science, educators worry that students who skip the mental struggle that is necessary for problem solving won't develop the skills they need to advance in upper-level classes and careers. That's why Schaffer, the Cornell professor, introduced the oral defense in his biomedical engineering class. He requires students to sign up for 20-minute sessions of Socratic-style questioning after submitting written problem sets, which are assigned several times each semester. With a class of 70 students, Schaffer splits the job with his teaching assistants. They no longer grade the written problem sets, just the oral defenses. He calls it "incentivizing" his students to do the work, or at least understand it enough to explain. Schaffer's class is highlighted in a new "Oral Assessment Workshop" offered by Cornell's Center for Teaching Innovation. Other examples at Cornell: a religious studies professor who now holds 30-minute "final conversations" with students instead of a final exam; and another engineering course where the professor gives four-minute mock interviews to each student in a 180-person class. Skeptics point out oral exams can be unsettling for students who are shy or have serious anxiety, but clarifying the format ahead of time and starting with softball questions can help, says Carolyn Aslan, who leads Cornell's oral exam training. "Sometimes it's actually good to get that quiet student one-on-one, and you finally get to hear from them. Sometimes that is the breakthrough," Aslan says. Several of Schaffer's students say they felt nervous at first but ended up preferring the oral exam. "I honestly liked it a lot," says Cornell junior Olivia Piserchia, a biomedical engineering major. She initially found the oral defense nerve-wracking but came to value the one-on-one time with instructors. It kept her from feeling lost in a large class and helped her build the skill of articulating her technical knowledge, as she would need to in a job. "Having that live check-in holds you accountable," says Piserchia. "It's a lot harder to look people in the eyes and say out loud, 'I don't know this.' And, that makes you realize, 'I should study this.'" ___ The Associated Press' education coverage receives financial support from multiple private foundations. AP is solely responsible for all content. Find AP's standards for working with philanthropies, a list of supporters and funded coverage areas at AP.org.