For millions of smartphone users worldwide, dictating a search query or a quick text message using voice commands has long been an everyday convenience. However, for the estimated 70 million Deaf and hard of hearing people globally who use sign languages, digital interaction has largely remained tied to traditional typing.
In a major development for accessible technology, Google DeepMind has launched SL2T (sign-language-to-text), an artificial intelligence model that allows users to perform web searches, write messages, and navigate their devices using sign language.
According to details released on the Google DeepMind Blog, the technology is debuting across Gboard and Live Transcribe on Pixel 11 devices, initially translating American Sign Language (ASL) directly into written English.
How the Technology Works
Unlike voice transcription, which maps spoken sounds into words in the same language, sign language recognition requires visual translation between entirely separate linguistic systems. Sign languages carry distinct grammatical rules, syntax, and facial expressions that do not mirror written text word for word.
To address these complexities, the engineering team at Google
DeepMind designed a multi-layered approach:
Privacy-First Tracking: Rather than storing or transmitting raw video footage, an on-device tracking system reads key geometric points across the user's hands, face, and torso. These numerical coordinates are processed for translation, while the original video feed is deleted immediately.
Direct Translation: SL2T translates the movement coordinates directly into fluent text sentences, capturing subtle facial cues and spatial movements without relying on intermediate annotations.
Extensive Dataset: The model was trained on more than 100,000 hours of data across 50 sign languages, with roughly a quarter dedicated to American Sign Language.
Designed for Everyday Use
To ensure the system works reliably outside laboratory environments, developers optimized the AI for common real-world conditions. This includes accurate recognition for one-handed signing-allowing users to sign with one hand while holding their phone with the other-as well as equal performance for left-handed signers.
The project was developed in close collaboration with Deaf researchers and advocacy groups, led by an internal advisory committee to ensure the tool met the practical needs of its target audience.
Looking Ahead
The deployment of SL2T on Pixel 11 marks the first time a sign-to-text AI model has been integrated directly into everyday consumer software at scale. Google plans to expand the feature to additional smartphone models and introduce support for further international sign languages in future updates.
