AI and Eye-Tracking Breakthrough: Pinpointing Exact Moments of Learning in Educational Videos

Tech can tell exactly when in videos students are learning

A new study combines eye tracking and artificial intelligence to identify the exact moments in an educational video that matter for learning in children. The study could also predict how much children understood from the video based on their eye movements while they were watching it. The research is preliminary, but it provides promise for some exciting breakthroughs in video education, said Jason Coronel, lead author of the study and associate professor of communication at The Ohio State University. "Our ultimate goal is to build an AI system that can tell in real time whether a viewer is understanding or not understanding what they are seeing in an educational video," Coronel said. "That would give us the opportunity to dynamically adjust the content for an individual person to help them understand what is being taught." Coronel conducted the research with Matt Sweitzer, Alex Bonus, Rebecca Dore, and Blue Lerner, an interdisciplinary team of experts in eye tracking, machine learning, and children's media, all affiliated with Ohio State. The study, published in the Journal of Communication, involved 197 children aged 4 to 8 who watched a four-minute composite video from the popular YouTube series "SciShow Kids" and "Learn Bright." The video taught children about camouflage in animals. Eye-tracking allowed researchers to measure attention to the video in real time, which is critical for learning, Coronel said. After watching the video, children were asked a series of questions to determine what they had learned about camouflage. (Before they watched, the children answered questions to assess their baseline knowledge.) An AI analysis of the eye-tracking results identified points in the video that are related to whether the children were able to answer the questions about camouflage correctly. For example, one key point was near the beginning when the video host asked children to help her find her anthropomorphic sidekick, named Squeaks. "Our machine learning and eye-tracking data indicate that children's eye movements during this early moment are among the strongest predictors of their overall understanding of the video," the study authors wrote. "One possibility, then, is that kids who follow the cue (to help find Squeaks) with focused attention become more engaged and better prepared to understand more complex concepts introduced later." The analysis identified seven key moments in the video where noticeable shifts in children's eye movements were more strongly linked to how well they understood the concept of animal camouflage. One study co-author, Alex Bonus, who has expertise in children and media, noted that the seven time points lined up with substantial changes in the video's educational content -- what researchers call "event boundaries." These boundaries are when people perceive that one meaningful experience is ending and a new one is beginning. For example, one event boundary came when the narrator began to define camouflage explicitly and paired the explanation with a visual display of the word. Coronel emphasized that these results are preliminary and there is a lot yet to understand about what happens at the critical points of a video when learning seems enhanced. "But this method has the potential to help experts design messages with event boundaries that enhance learning," he said. The findings are especially relevant now, as eye-tracking technology becomes less expensive and more common, Coronel said. That, along with advances in AI, makes possible a future where video learning is truly individualized. Right now, for example, it can take days or weeks for teachers to find out if students are understanding their lessons. Often, teachers don't find out until the next test or quiz. "Imagine a future where eye tracking can tell instantaneously when a person is not understanding a concept in a video lesson, and AI dynamically changes the content to help," he said. "Maybe the video can offer a different example or way of explaining the concept. This could make instruction more personalized, effective and scalable."

Tech Can Tell Exactly When in Videos Students Are Learning | Newswise

COLUMBUS, Ohio - A new study combines eye tracking and artificial intelligence to identify the exact moments in an educational video that matter for learning in children. The study could also predict how much children understood from the video based on their eye movements while they were watching it. The research is preliminary, but it provides promise for some exciting breakthroughs in video education, said Jason Coronel, lead author of the study and associate professor of communication at The Ohio State University. "Our ultimate goal is to build an AI system that can tell in real time whether a viewer is understanding or not understanding what they are seeing in an educational video," Coronel said. "That would give us the opportunity to dynamically adjust the content for an individual person to help them understand what is being taught." Coronel conducted the research with Matt Sweitzer, Alex Bonus, Rebecca Dore, and Blue Lerner, an interdisciplinary team of experts in eye tracking, machine learning, and children's media, all affiliated with Ohio State. The study, published today in the Journal of Communication, involved 197 children aged 4 to 8 who watched a four-minute composite video from the popular YouTube series "SciShow Kids" and "Learn Bright." The video taught children about camouflage in animals. Eye-tracking allowed researchers to measure attention to the video in real time, which is critical for learning, Coronel said. After watching the video, children were asked a series of questions to determine what they had learned about camouflage. (Before they watched, the children answered questions to assess their baseline knowledge.) An AI analysis of the eye-tracking results identified points in the video that are related to whether the children were able to answer the questions about camouflage correctly. For example, one key point was near the beginning when the video host asked children to help her find her anthropomorphic sidekick, named Squeaks. "Our machine learning and eye-tracking data indicate that children's eye movements during this early moment are among the strongest predictors of their overall understanding of the video," the study authors wrote. "One possibility, then, is that kids who follow the cue (to help find Squeaks) with focused attention become more engaged and better prepared to understand more complex concepts introduced later." The analysis identified seven key moments in the video where noticeable shifts in children's eye movements were more strongly linked to how well they understood the concept of animal camouflage. One study co-author, Alex Bonus, who has expertise in children and media, noted that the seven time points lined up with substantial changes in the video's educational content - what researchers call "event boundaries." These boundaries are when people perceive that one meaningful experience is ending and a new one is beginning. For example, one event boundary came when the narrator began to define camouflage explicitly and paired the explanation with a visual display of the word. Coronel emphasized that these results are preliminary and there is a lot yet to understand about what happens at the critical points of a video when learning seems enhanced. "But this method has the potential to help experts design messages with event boundaries that enhance learning," he said. The findings are especially relevant now, as eye-tracking technology becomes less expensive and more common, Coronel said. That, along with advances in AI, makes possible a future where video learning is truly individualized. Right now, for example, it can take days or weeks for teachers to find out if students are understanding their lessons. Often, teachers don't find out until the next test or quiz. "Imagine a future where eye tracking can tell instantaneously when a person is not understanding a concept in a video lesson, and AI dynamically changes the content to help," he said. "Maybe the video can offer a different example or way of explaining the concept. This could make instruction more personalized, effective and scalable."