Using artificial intelligence in an interactive, tablet-based video game has the potential to revolutionise the way children undertake speech therapy, by making it more accessible and fun.
University of New South Wales electrical engineer and co-founder of Say66 Dr Beena Ahmed recently won a Women in AI Award for her work in the development of the speech therapy tool.
Ahmed’s award follows 12 years’ research into the topic of automatically detecting pronunciation errors in disordered speech and a career-long passion for applying engineering to speech and healthcare, specifically using signal processing, machine learning and wearable sensors.
Real-time feedback
More than 50 per cent of children with communication disorders receive no, or insufficient treatment, Ahmed told create.
“This problem is exacerbated in today’s COVID-19 world, as many children struggle to engage in non-contact virtual therapy,” she said.
“Say66’s engaging video game, Say Bananas, normalises speech practice, as children’s peers are playing the same style of games, and also reaches under-served children constrained by geography or funding.”
It also does much more to assist children and speech-language pathologists (SLPs).
Say Bananas uses automated AI tools to analyse children’s speech as part of intensive, high-fidelity practice and give them real-time feedback through a pronunciation evaluation algorithm.
A custom-built, offline speech analysis algorithm also analyses children’s speech in detail for pronunciation and lexical stress errors, Ahmed said.
For SLPs, AI tools can provide detailed assessment reports, while a web-based clinician interface allows home therapy practice to be remotely monitored, and reduces the need for more frequent one-on-one sessions.
While mobile games developed by therapists have previously been used to deliver speech therapy exercises for children, these have lacked feedback mechanisms, said Ahmed, who gained a Bachelor of Science in Electrical Engineering from the University of Engineering and Technology Lahore in Pakistan.
“Games integrated with automatic speech recognition (ASR), where speech productions control gameplay, offered the solution,” she said.
Changing children’s lives
It was an interaction with a speech therapist that sparked Ahmed’s interest in children’s speech difficulties, and the development of AI-powered speech therapy tools.
“[The therapist] shared how large SLPs’ caseloads are and how hard it is to keep children engaged,” Ahmed said.
“Ten per cent of children experience a speech sound disorder that makes their speech unintelligible, negatively impacting communication development, friendships and reading.”
Large amounts of speech data, needed to train deep machine learning algorithms, can be difficult to obtain from children (often due to privacy issues). Ahmed overcame this problem by turning mispronunciation detection into an anomaly detection problem, using algorithms trained with only correctly pronounced speech data.
To date, clinical trials of Say Bananas have shown equivalent progress compared to face-to-face therapy with an SLP, at one-fourth of the cost. And the feedback from parents involved in trials has been overwhelmingly positive.
Ahmed said she was confident that the AI-enabled video game would directly impact children’s learnings in an affordable, reliable way, making speech therapy accessible across the country, including in remote communities.
Say Bananas is expected to launch to the market by the middle of this year.