Google's Pixel 11 Introduces Sign Language to Text Translation for 70 Million Deaf Users

Google Launches Sign Language to Text AI Model on Pixel 11

Google DeepMind introduced sign-language-to-text (SL2T), a breakthrough AI model that translates American Sign Language into text, debuting on the Pixel 11 series at the Made by Google event2

2

5

5

. This marks the first time sign language AI at this scale has reached a consumer product, addressing a gap that has left an estimated 70 million deaf and hard of hearing users behind in the AI revolution2

2

5

5

. The Pixel 11 Pro Fold, starting at $1,899 and available for pre-order with sales beginning August 20, includes a free six-month Google AI Pro subscription1

1

.

The sign-to-text dictation feature enables deaf and hard of hearing users to sign directly to their phone's camera anywhere they would normally type, transforming how they interact with technology. Users can sign to search the web, draft messages or documents, and ask Gemini to solve queries or execute tasks2

2

3

3

. In Live Transcribe, users can sign responses during face-to-face conversations instead of typing back and forth2

2

4

4

. According to Google's testers, signing in American Sign Language is faster, more natural, and more delightful than typing in English2

2

.

How the On-Device Computer Vision System Works

The sign-language-to-text (SL2T) model operates through an on-device computer vision system called MediaPipe Holistic, which tracks 130 points across a person's face, body, and hands3

3

5

5

. This on-device processing turns video into a moving map of geometric coordinates, immediately discarding the raw footage while sending only those coordinates to Google's servers for real-time translation3

3

4

4

. Google does not retain logs of what users sign or the text the system produces unless someone explicitly agrees to participate in a model evaluation study3

3

.

Unlike earlier approaches that converted signs into written labels known as "glosses" before translation, SL2T translates directly from landmarks into English text3

3

5

5

. This direct translation approach allows the model to capture facial expressions, physical positioning, and other movements that carry meaning, removing artificial vocabulary limits3

3

. The model can "see" and understand physical movement across a user's hands, arms, torso, head, and face simultaneously1

1

.

Training Data Spans 100,000 Hours Across 50+ Languages

Google trained sign-language-to-text (SL2T) on more than 100,000 hours of sign language data spanning over 50 sign languages, with roughly one-quarter of the data in American Sign Language1

1

4

4

. The company trained across multiple sign languages to learn shared structural patterns, enabling it to significantly outperform earlier sign language models5

5

. SL2T achieved a score of 70 BLEURT on the FLEURS-ASL benchmark, backed by solid performance data5

5

.

The model includes optimizations to reduce latency, hallucination prevention mechanisms for non-signing movements, and support for both left-handed signers and one-handed signing4

4

5

5

. Left-handed signers make up about 10 percent of users, while one-handed signing is common when someone is holding their phone in the other hand4

4

.

Using Sign-to-Text Dictation in Gboard and Live Transcribe

To activate the feature on the Pixel 11 Pro Fold, users open the Translate app and tap the sign language icon in Gboard's top toolbar1

1

. The phone uses its front camera to recognize what the user is signing, and the transcription displays on the cover screen so the other person can easily read it and respond1

1

. The AI-powered accessibility tool works anywhere users would normally type, transforming the phone from a one-way listening tool into one that supports natural, two-way conversation during spontaneous, face-to-face interactions4

4

.

The Pixel 11 also supports transcribing speech to text, providing a text field for typing responses or questions for the other person to read1

1

. This combination creates a more complete accessibility solution for deaf and hard of hearing users communicating with people who don't know sign language4

4

.

Current Limitations and Not a Replacement for Human Interpreters

The first version has significant limitations that users should understand. Google DeepMind's report states the model can struggle with regional signs, slang, complex American Sign Language grammar, rapid fingerspelling, and meanings conveyed primarily through facial expressions or head movements3

3

. It may also generate "ghost text," or words the user did not sign, when someone pauses or another person enters the camera frame3

3

.

Accuracy can decline in low light, when a signer is partially outside the frame, or when there is insufficient contrast between a person's hands, face, clothing, and surroundings3

3

. The model has not been formally evaluated on people under 18 or extensively tested across signers with motor disabilities3

3

.

Google and its AI Sign Language Advisory Committee describe sign-language-to-text (SL2T) as "strictly an assistive tool for low-stakes communication" that does not fulfill legal requirements for reasonable accommodations3

3

5

5

. It is not designed for medical appointments, legal proceedings, classrooms, job interviews, government hearings, or other settings where an error could carry serious consequences3

3

. The report warns institutions against using the software to replace qualified human interpreters or avoid their accessibility obligations3

3

.

Expansion Plans and Advisory Committee Guidance

American Sign Language transcription is currently available on the Pixel 11 series in select countries and only supports ASL-to-English translation1

1

. Google confirmed that support for more devices and additional languages is coming soon at no additional cost3

3

. The company plans to expand the model to cover additional sign languages and develop models for sign language generation5

5

.

Google enlisted Deaf employees, outside testers, advocacy organizations, professional interpreters, and sign language experts to guide the model's development3

3

. The company established the AI Sign Language Advisory Committee in partnership with deaf organizations and sign language experts to help guide responsible deployment5

5

. Google plans to continue working with this committee as it expands the technology to additional sign languages and develops AI that can generate sign language3

3

. Whether sign-language-to-text (SL2T) holds up in fast, everyday signing beyond benchmark conditions will determine if it becomes a genuine communication tool for the global deaf community.