MIT's ultrasound wristband lets wearers control robotic hands with their own hand movements

MIT Engineers Develop Ultrasound Wristband for Real-Time Hand Tracking

MIT engineers have unveiled a wearable ultrasound wristband that addresses one of robotics' most persistent challenges: capturing the nuanced dexterity of human hands. The device produces continuous ultrasound images of the wrist's muscles, tendons, and ligaments, paired with an AI algorithm that translates these images into precise finger and palm positions 1

1

. Published in Nature Electronics, the research demonstrates how ultrasound imaging can track complex hand movements involving 34 muscles, 27 joints, and over 100 tendons and ligaments that coordinate every gesture 3

3

.

The wristband represents a significant advance over existing approaches to hand tracking. Current methods rely on camera systems prone to visual obstacles, sensor-laden gloves that restrict natural hand movements, or electrical muscle signals easily affected by environmental noise 4

4

. These electrical approaches also lack the sensitivity to distinguish subtle changes in movements, such as the continuous path between pinched and separated fingers. By contrast, the ultrasound wristband can identify all 22 degrees of freedom the human hand is capable of, enabling it to differentiate all 26 letters in American Sign Language 2

2

.

How the Wearable Technology Translates Wrist Muscles and Tendons into Robotic Action

"The tendons and muscles in your wrist are like strings pulling on puppets, which are your fingers," explains co-author Gengxi Lu. "So the idea is: Each time you take a picture of the state of the strings, you'll know the state of the hand" 1

1

. The device incorporates an ultrasound sticker the size of a smartwatch, paired with onboard electronics about as small as a cellphone, though researchers acknowledge the current version resembles more of a cyberpunk gauntlet than a sleek wearable 2

2

.

The AI algorithm was trained on ultrasound images carefully labeled by humans, enabling it to rapidly analyze incoming images and determine corresponding hand positions 2

2

. This approach builds on Zhao's previous work developing ultrasound stickers—miniaturized transducers paired with hydrogel material that safely adheres to skin 3

3

. The system proved precise enough to control robotic hands wirelessly, allowing wearers to manipulate robots to play piano tunes, shoot basketballs into desktop hoops, and perform intricate gestures in real-time 1

1

.

Applications in Virtual Reality and Training Humanoid Robots

"We think this work has immediate impact in potentially replacing hand tracking techniques with wearable ultrasound bands in virtual and augmented reality," says Xuanhe Zhao, the Uncas and Helen Whitaker Professor of Mechanical Engineering at MIT. "It could also provide huge amounts of training data for dexterous humanoid robots" 4

4

. The wristband enables gesture control in virtual environments, where wearers can pinch their fingers to zoom in and out on virtual objects without touching a screen 3

3

.

The team is now gathering hand motion data from users with different hand sizes, finger shapes, and gestures to build a comprehensive dataset 5

5

. This data could train humanoid robots in dexterity tasks such as performing surgical procedures, or enable more immersive interactions in video games and design applications 1

1

. For robotics researchers who have long struggled with tasks as seemingly simple as peeling a banana without squishing it, this technology offers a pathway to capture the complexity that makes human hands so versatile 2

2

.

Looking ahead, the MIT team plans to reduce the device's size while expanding their AI training to encompass a wider variety of volunteers. The goal is to finalize a wearable that anyone can use to remotely control robots or interact with virtual environments 2

2

. As machines continue to outperform humans in tasks like chess or operating in hazardous environments, this ultrasound wristband may finally give robots the one advantage they've been missing: human-like hand dexterity.