Voiceitt app allows people with speech impairments to converse with others and to control Alexa-enabled smart home devices using their own voices.
By Abigail Klein Leichman, ISRAEL21c
Danny Weissberg’s grandmother struggled to communicate after suffering a stroke. While visitors had difficulty understanding her impaired speech, her full-time caregiver gradually learned to understand it.
Working as a software developer in high-tech companies including Check Point, Radvision and HP since 1999, Weissberg spotted the potential of early machine-learning technology to mimic what the caregiver had done naturally.
He founded Voiceitt to harness that technology. The Ramat Gan-based company’s new app empowers individuals with non-standard speech to communicate with others and control smart-home devices using their own voice.
“About nine years ago, I saw a strong trend for human-machine interface. It was clear to me that voice and speech recognition would become big,” Weissberg tells ISRAEL21c.
A dedicated volunteer with the elderly and with Save a Child’s Heart, Weissberg saw this as an opportunity “to do something bigger to create an impact.”
“Giving people with disabilities the ability to communicate has the potential to change lives in a meaningful way with technology,” he says.
Discussions with speech therapists revealed that impaired speech impacts many children and adults with hearing impairment, autism, Down syndrome, neurological conditions or conditions affecting the throat or mouth.
Speech-recognition expert Stas Tiomkin, like Weissberg a graduate of the Technion-Israel Institute of Technology, was then researching machine learning at Hebrew University in Jerusalem. Their former professor introduced them.
“Stas told me all the major companies are developing speech recognition for standard speakers. He thought my idea was not only unique but feasible,” Weissberg recalls.
Personalized speech model
This was 2012, when voice assistants began appearing on the market.
“Voice interface became dominant with products such as Siri, Cortana, Alexa, Bixby and Google Assistant. We thought maybe we had something interesting to offer to these big corporates to make their products accessible to a wider population that needs these voice assistants the most,” says Weissberg.
Any atypical speech makes using voice assistants difficult, whether it’s slurred pronunciation, a heavy accent, or speaking a language that’s not supported by the technology.
“We started to develop a speech recognition engine that learns a person’s speech the way a person does, and builds a personalized speech model for him,” says Weissberg.
The big corporates liked the idea.
After initial funding and assistance from what is now the Israel Innovation Authority, EzraTech and angel investors in the United States and Israel, Voiceitt won Microsoft’s AI For Good competition and received a grant from its AI for Accessibility Fund.
“Microsoft CEO Satya Nadella has a personal connection because his son was born with cerebral palsy,” Weissberg explains.
Voiceitt received investment from Amazon through the Alexa Fund and support from the Alexa Techstars Accelerator in Seattle. A pilot in collaboration with the American Association of Retired Persons (AARP) followed along with investment from AARP. The successful pilot led to the integration of the Alexa API into Voiceitt’s application.
“That means a person who cannot use Alexa because of a speech disability can use Voiceitt to translate what he says and access Alexa,” says Weissberg.
“So this innovation is not just for person-to-person communication but also for smart homes and smart cars, for video conferencing with real-time closed captions, and many more scenarios.”
Weissberg points out that the biblical Moses, described as having some kind of speech impediment, communicated through his brother Aaron.
“Aaron was the one who could understand him. We’re like the Aaron of the 21st century.”
Opening doors
The Voiceitt experience begins with training the software to recognize an individual’s speech in any language.
Users build a personalized dictionary of specific words and phrases that the app learns to understand in up to five minutes per phrase.
“You train it for each command individually, like ‘Turn off the lights.’ It doesn’t extrapolate ‘Open the door’ from that,” explains Weissberg. “We are developing a support for fluent speech for more flexibility and that will be in English first. It should be ready in a year from now.”
One of the company’s Israeli testing sites, ALYN Pediatric and Adolescent Rehabilitation Hospital in Jerusalem, adopted Voiceitt to allow patients with atypical speech to interact with Alexa in Hebrew or Arabic.
“In collaboration with Voiceitt, we have been able to bring Alexa to even more customers,” said Peter Korn, Director of Accessibility at Amazon Lab 126.
“We strive to make all Amazon products and services as delightful and easy to use as possible for everyone. Voiceitt’s integration with Alexa helps us do just that, enabling customers with speech impairments to enjoy all that Alexa has to offer and help them live more independently.
Voiceitt has raised $17 million from Amazon Alexa Fund, M12 (Microsoft’s venture capital fund), AMIT Technion, Viking Maccabee Ventures, Connecticut Innovations, Cahn Capital, Quake Capital Partners, Dreamit Ventures and others. The company has a subsidiary in the United States.
The app –available for iOS devices — comes at a time when isolation caused by the pandemic has left many individuals with speech and motor impairments separated from loved ones or caregivers.
By offering the Voiceitt app at no cost until this fall, Weissberg and Tiomkin and third cofounder Sara Smolley aim to “mitigate some of the hardships created by the pandemic, for as many people affected as possible.”