BioCoach AI delivers real-time biomechanical feedback to prevent workout injuries at home

New BioCoach AI provides real-time biomechanical feedback for exercises

As any athlete will tell you: perfect practice makes perfect. But for individuals who do not have regular access to coaches or trainers, maintaining good form can be tricky. In fact, during the Covid-19 pandemic when many people were exercising at home, the U.S. Consumer Product Safety Commission reported a 48% rise in injuries related to at-home exercise. In hopes of preventing some of these injuries and extending the expert guidance of coaches, researchers from Drexel University and Michigan State University have developed a prototype of a program that uses artificial intelligence and computer vision to analyze video and provide form coaching in real time. The program, which integrates biomechanical modeling with computer vision and a vision-language model, is designed to provide live, personalized feedback and explanations of the guidance it offers during an exercise - a feat that has proven to be challenging for most fitness coaching apps. The researchers published their work ahead of presenting their prototype, called BioCoach, at the Conference on Computer Vision and Pattern Recognition, hosted by the Institute of Electrical and Electronics Engineers and the Computer Vision Foundation in June. "Many people who exercise at home with videos and apps don't get high-quality assessment of their movements," said Feng Liu, PhD, an assistant professor in Drexel's College of Engineering and Computing, who led the research. "Feedback is often too generic or simply encouragement but no actual form coaching. Our goal with BioCoach is to provide timely, specific cues grounded in body motion, closer to the kind of guidance a knowledgeable coach would give." Feng's Visual Intelligence Lab at Drexel applies advanced computer vision, machine learning and 3D human-body modeling to study problems in exercise coaching, clinical gait assessment and classroom education. To prepare BioCoach, the team started with an exercise-video coaching benchmark - the publicly available Qualcomm Exercise Video Dataset (QEVD), which includes hundreds of hours of exercise footage along with time-stamped coaching feedback. The feedback included only short coaching comments, such as "lower your body more." So the researchers created a new version by re-annotating it with more detailed biomechanical targets, "increase elbow flexion to 90 degrees at the bottom," for example. They also added short rationale for the guidance, such as "increase hip/knee flexion to distribute load." In all, the team added more than 2,400 notes to over 200 videos used to train and test BioCoach. These annotations helped to prepare the large language model that provides coaching and guidance to the user. And because the time stamps were preserved in their annotated dataset, this new benchmark would enable the researchers to evaluate not only the guidance the system was offering, but also whether it responded at the right moment. With the improved exercise video feedback dataset in hand, the team designed BioCoach to analyze each video through two complementary streams of information in order to access and deliver the proper guidance to the user. One stream captures visual appearance and motion patterns using 3D convolutional neural network - a deep-learning program adept at identifying individual objects in images and videos. The other allows BioCoach to estimate 3D skeletal movements and body shape, giving the program access to information about joint angles, ranges of motion and exercise phases. With access to these information streams, BioCoach is able to access structured biomechanical data unique to each joint. This means before providing feedback, it first identifies the joints most relevant to each exercise - for example, the hips, knees and ankles for squats or the shoulders, elbows and wrists for push-ups - so that it can provide more detailed guidance. Through this process, the program is also able to use body-shape information and movement-quality analysis to provide the structured information its language model translates into specific, biomechanics-based feedback. Our goal was to build a system that does more than look at pixels and generate a generic comment. BioCoach exposes the model to 3D motion, joint angles and exercise-specific constraints, so the feedback can point to a concrete movement issue and explain why it matters." Feng Liu, PhD, Assistant Professor, Drexel's College of Engineering and Computing After preparing the program, the team set out to test it against the top competition - video-language AI programs by research and development teams from NVIDIA, ByteDance, Alibaba, Salesforce, OpenAI, Shanghai Jiao Tong University, Chinese University of Hong Kong, Peking University and Peng Cheng Laboratory in China, and the Massachusetts Institute of Technology. They tested the programs by showing each program a number of exercise videos - some from the original QEVD set and some that the team had annotated. The responses of each program were compared to those offered in the original QEVD dataset as well as those added by the researchers, with scoring based on how timely, accurate and detailed they were. In responding to videos from the original dataset, BioCoach outperformed its nearest competition, Stream-VLM - a program created by researchers from MIT and NVIDIA - in text quality and judged correctness, while its timing score was close but slightly lower. But it outpaced Stream-VLM across all metrics when its feedback was graded against that from the dataset with more specific annotations, showing particular improvements in biomechanical correctness and detailed, anatomy-specific feedback. The researchers suggest that these results show that adding explicit 3D kinematics and biomechanical context can improve the quality and interpretability of real-time exercise feedback without substantially reducing responsiveness. "It was encouraging to see that BioCoach was able to perform so well against programs made by some of the top researchers and companies in the AI field," Feng said. "This is still a prototype, but it shows how combining computer vision with structured biomechanical reasoning can make AI coaching systems more useful and easier to inspect." The team plans to continue its work by enhancing the program so that it can estimate joint reaction forces and muscle activation patterns from videos in order to detect slight compensatory movements that could result in injuries during exercise. "We believe this work could ultimately support exercise and physical-therapy apps that extend the expertise of human coaches and trainers between in-person sessions," Liu said. "A future system could help users receive more specific, timely feedback when they practice on their own, while still keeping human experts in the loop." Source: Drexel University Journal reference: Ji, Y., et al. (2026) From 3D Pose to Prose: Biomechanics-Grounded Vision--Language Coaching. Conference on Computer Vision and Pattern Recognition. DOI: 10.48550/arXiv.2603.26938. https://arxiv.org/abs/2603.26938

AI fitness coach senses the muscle mechanics as you exercise and prevents rookie injuries

Most fitness apps offer encouragement dressed up as coaching, but BioCoach offers anatomy-specific corrections, and I could see it becoming a smartphone app real soon. During the pandemic, the US Consumer Product Safety Commission recorded a 48% spike in at-home exercise injuries. You might think that the culprit was bad equipment, but it was bad form. People had no coach around to correct it. Researchers at Drexel University and Michigan State University have built a prototype that addresses exactly that problem, in real time, using your phone camera, and there's real potential for it to become a legitimate fitness app in future (via Tech Xplore). What is the system called and how does it work? The system, called BioCoach, was presented at the Conference on Computer Vision and Pattern Recognition in June 2026. It uses AI and live video (via a camera) to watch you exercise, analyze your body mechanics, and deliver specific, biomechanics-based corrections. Recommended Videos To do this, the system processes video through two parallel streams: first uses a 3D convolutional neural network to capture your visual appearance and body movement patterns, while the second reconstructs your skeleton in three dimensions, analyzing your joint angles, range of motion, and the phase of the movement you're in. Before offering you feedback, BioCoach identifies which joints are most involved in the exercise you're performing. For instance, if you're performing push ups, it will specifically monitor your shoulders, elbows, and wrists, offering personalized corrections. And I'm not talking about the generic "keep your back straight" comments that most fitness apps offer. The prototype goes above and beyond, offering anatomically precise guidance like "increase elbow flexion to 90 degrees at the bottom." How did it perform against the competition? The research team has trained BioCoach on Qualcomm's Exercise Video Dataset, with over 200 re-annotated videos and over 2,400 new notes, to teach BioCoach to explain not just what to fix, but why it matters. BioCoach has already been tested against similar programs from Nvidia, ByteDance, Alibaba, Salesforce, OpenAI, and MIT, among others. It outperformed Stream-VLM, which is a program from MIT and Nvidia, on text quality and judged correctness. It showed improvements in anatomy-specific feedback accuracy as well. For now, the system is still a prototype, but the team is working on adding the ability to estimate joint reaction forces and muscle activation patterns, all from a video feed. The research was supported by the National Science Foundation, and this is why I strongly believe that BioCoach could be developed in a revolutionary smartphone app, which offers personalized corrective measures and encourages the right form and posture, preventing painful injuries and sustainable workout programs for people, which works both indoors and outdoors. BioCoach is more advanced than most AI-based fitness coaches available To give you some context, both Apple Fitness+ and Mirror offer video-based workout programs, but the feedback is pre-recorded and not dynamic like what BioCoach offers. Peloton's hardware offers a Movement-Tracking Camera that counts reps and flags issues, but it requires dedicated equipment like Bike+, Tread+, or Row+, and doesn't explain the reasoning behind the form corrections and how they can benefit you. Similarly, Google's Health Coach and Samsung Health analyze biometric signals like heart rate and activity cadence to suggest certain improvements, but they can't see you moving, and therefore, don't provide any guidance for your form. BioCoach, in contrast, is the first system to combine 3D skeletal reconstruction with a language model that explains the mechanical consequence of each correction. If it ever reaches your phone as a consumer app, which I truly hope it does, it could make genuinely expert coaching accessible to anyone with a camera.