Humanoid robot masters tennis using AI that learns from imperfect motion data

Watch: Humanoid robot gets surprisingly good at tennis

The remarkable Unitree G1 continues to astound with its abilities - now it's learned to play tennis This ain't teleoperation. Chinese researchers have tested a new, much quicker and easier method of teaching robots to play tennis, and the results look like a breakthrough in machine learning and real-world AI. What does it take to be a decent sportsperson? Highly accurate perception, for a start - plus a lot of physical dexterity, excellent predictive abilities, fast reflex reactions, a sixth sense for angles, and no small amount of technique specific to the given sport. The lattermost has been a challenge for robotics researchers; in tennis, as in most sports, wearable motion capture tech struggles to deal with how far tennis players run during a rally, and also can't yet read the tiny nuances of wrist angle and whatnot that separate a good shot from a bad one. It's far too dynamic a situation to make teleoperation an option. And trying to divine this stuff from multi-camera TV footage using AI training software like nVidia's Vid2Player3D... Well, according to Zhang et al, authors of a new study, that's a "complex pipeline" that "may require substantial expertise and engineering efforts." The team's new LATENT system goes back to motion capture, but only for the building blocks of technique, and it's designed to work with imperfect data. Effectively, in the current experiment, the researchers took some five hours' worth of motion capture data, in which human sportspeople demonstrated the "primitive skills" required for tennis: forehands, backhands, sideways shuffles and crossover steps, executed within a fraction of the area of a full-sized tennis court. They crunched these motion captures to create a repertoire of human-like 'motion spaces,' then loaded these basic skills into the robots - in this case, Unitree's G1 humanoid, which you've seen all over the place doing everything from dance numbers to kickboxing, and which is now available from a pretty wild starting price of ~US$13,500. Effectively, the LATENT system then more or less told the robots 'ok, there's how you should move. Now, using motions somewhat similar to those, your task is to see a tennis ball coming, and use your racket to hit it back over the net. Success is a ball landing on the opposite side of the court, within the white lines.' With those basic skills and motions to choose from, the robots were then able to experiment with all the rest of the details; angles, timing, which movements to use for which purposes and when to move outside of the trained motions. The vast majority of this learning was done at greatly accelerated speed in simulation. And the real-world results? Well, the G1 returned forehands at around 90% success and backhands at just under 80%, and looks remarkably agile and fluid and... An awful lot like a tennis player while doing it. Check it out: Clearly, it's not ready for Wimbledon. Indeed, it's not ready for any sort of competitive match yet. But for an early-days effort, this represents remarkable progress. It looks to me like it won't be long before a 10-grand Chinese robot will make a pretty dang decent tennis training partner, and the path is gradually being paved toward a world where the best professional tennis players have about as much chance of beating these things as a chess grandmaster has of beating an AI opponent. Of course, pro tennis player isn't exactly the kind of routine, repetitive job people have been desperately hoping robots will take over. But robots will get some of the same benefits humans do out of sport - they'll learn to master their bodies under extreme circumstances, dealing with complex and highly dynamic situations, in ways that'll be useful in more practical tasks... Like, say, beating protestors about the head with Agassi-level style and flair!

This Video of a Humanoid Robot Playing Perfect Tennis Is Extremely Impressive

Sports are also no match for the agile automatons, from shooting hoops to live kickboxing matches in front of a crowd. Now, Chinese AI robotics company Galbot has designed software that teaches a Unitree G1 humanoid robot how to perform a veritable feat: effectively playing tennis, allowing it to keep its own in a match-up against a human engineer. A video the company posted to social media shows the white robot holding up what appears to be an unmodified tennis racket, which it uses to easily return the ball by shuffling across the court. It's yet another impressive demonstration of how far the tech has come -- but whether the robot will be able to keep up with the likes of Novak Djokovic or Serena Williams any time soon remains to be seen, given the lighthanded volleys from the human engineers. "For the first time, a humanoid robot can sustain high-dynamic, long-horizon tennis rallies with millisecond-level reactions, precise ball striking, and natural whole-body motion," Galbot claimed in its post. "This marks a leap from mechanical motion imitation to intelligent, decision-driven athletic interaction." To teach their robot how to play tennis, the company's engineers built a system that had to rely on "imperfect human motion data" that consists "only of motion fragments that capture the primitive skills used when playing tennis rather than precise and complete human-tennis motion sequences from real-world tennis matches," according to a yet-to-be-peer-reviewed paper. "Our key insight is that, despite being imperfect, such quasi-realistic data still provide priors about human primitive skills in tennis scenarios," they found. "With further correction and composition, we learn a humanoid policy that can consistently strike incoming balls under a wide range of conditions and return them to target locations, while preserving natural motion styles." The engineers argue that the system could have uses beyond tennis as well. "Although this work primarily focuses on the tennis return task, the proposed framework has the potential to generalize to a broader range of tasks where complete and high-quality human motion data are unavailable," they concluded.

Super agile humanoid robot seen playing tennis with humans in uncanny...

Researchers have taught a humanoid robot to play tennis with humans -- and it can hold its own. Chinese AI robotic company Galbot designed software to teach a Unitree G1 humanoid robot to play tennis against a human engineer. The company posted a video to social media showing a white robot holding what appears to be an unmodified tennis racket and using it to return the ball as it shuffles across the court. "Your humanoid tennis player is here!" Galbot wrote on X. "For the first time, a humanoid robot can sustain high-dynamic, long-horizon tennis rallies with millisecond-level reactions, precise ball striking, and natural whole-body motion." "This marks a leap from mechanical motion imitation to intelligent, decision-driven athletic interaction." The software is dubbed LATENT (Learning Athletic Humanoid Tennis Skills from Imperfect Human Motion Data), and the company claims it's the world's first real-time whole-body planning and control algorithm for athletic humanoid tennis. According to a yet-to-be-peer-reviewed paper, the system had to rely on "imperfect human motion data" consisting only of "motion fragments that capture the primitive skills used when playing tennis" rather than clean motion capture from "real-world tennis matches." The short fragments of human movement used were made up of things like forehand swings, backhand strokes and basic footwork. These motion fragments become a library of movement building blocks which the robot stitches together and figures out how to combine them in real time. When it comes to wrist control, the robot's high-level controller directly adjusts the wrist during play rather than using the "imperfect" data. The robot can sustain multi-shot tennis matches with humans, reacting to balls traveling over 15 meters per second, which is about 33.5 miles per hour, and manages to produce coordinated strokes and footwork. The movements produced look relatively natural -- especially for a robot. It's not exactly fluid like a human, but it's not rigid and robotic either. "Our key insight is that, despite being imperfect, such quasi-realistic data still provide priors about human primitive skills in tennis scenarios," the researchers found. "With further correction and composition, we learn a humanoid policy that can consistently strike incoming balls under a wide range of conditions and return them to target locations, while preserving natural motion styles." In simulation tests, the system achieved up to 96% success in forehand shots. However, the engineers said that the software could be useful beyond the ability to play tennis. "Although this work primarily focuses on the tennis return task, the proposed framework has the potential to generalize to a broader range of tasks where complete and high-quality human motion data are unavailable," they noted. If a robot can learn a complicated physical skill like tennis from imperfect data, it suggests that similar approaches can work for real-world tasks as well. Earlier this year, it was reported that bots resembling humans could fold laundry, answer doors and even get you coffee.