Smartphones could soon be able to “understand” and translate sign language, thanks to new AI technology from Google.
The search giant has developed new systems that could make smartphones better equipped to understand complex hand signals and gestures. This technology has been hailed by tech insiders as well as those from the hearing-impaired community as the first step towards making devices that can translate sign language.
Previously, smartphones were only able to process the most rudimentary of hand signals, with flicks of the wrist or bends of fingers confusing the system – often because they hid other parts of the hand. However, this latest iteration graphs 21 points across the hand, fingers and wrist, to better map out any gesture from any angle. This means smartphones can now identify when someone’s fingers are touching or when their wrists are turned, for example.
Whilst this technology has already been made available in one guise on PC devices, it’s the first time that smartphones have been able to competently identify complex hand signals.
To prove its effectiveness, researchers showed the technology identifying a number of sign language words that grew in complexity. Whilst it started with numbers one to five (shown by raising certain fingers) it quickly moved on to show the system could differentiate between ‘rock’ (the widely known ‘devil horns’ hand gesture) and ‘Spiderman’ (devil horns again but this time with the thumb pointing outwards).
Though Google engineers Valentin Bazarevsky and Fan Zhang stressed that the technology was still in its infancy, they promised to open it up to the community so greater development could be done.
“We plan to extend this technology with more robust and stable tracking, enlarge the amount of gestures we can reliably detect, and support dynamic gestures unfolding in time,” they wrote on Google’s AI Blog. “We believe that publishing this technology can give an impulse to new creative ideas and applications by the members of the research and developer community at large. We are excited to see what you can build with it!”
