The onset of numerous neurological diseases can make communication difficult. Although research on solving these problems is ongoing, this is stymied by the lack of a relevant, sizeable language dataset to help researchers unlock what those affected are trying to say. Two CUHK professors are collaborating on technology solutions for Cantonese speakers in Hong Kong who suffer from dysarthria, which affects the articulation of sounds and words, and have since expanded their research to neurological diseases such as dementia. Their cross-disciplinary research combines multilingual speech pro-cessing, artificial intelligence, neuroscience and language learning.
Neurological diseases can make communication by speech a struggle for many. When the muscles needed for speech become weak or hard to control, those affected begin to slur their words in a disorder called dysarthria. Treatment is difficult because, outside of Hong Kong, there is no body of relevant research in the Cantonese language to draw upon. Professor Helen Meng Mei-ling and Professor Patrick Chun Man Wong of CUHK have been working together for more than a decade on speech and language processing. While Professor Meng deals with the digital and AI-related aspects of their projects, Professor Wong brings neuroscience and linguistics know-how in a cross-disciplinary approach to the challenge. Their collaborative efforts have resulted in a variety of projects that cater to what they describe as “development across the lifespan.”
The professors began by developing artificial intelligence models to recognise symptoms of dysarthria from speech samples, but have since broadened the scope of their research, hoping to distinguish signs of cognitive decline before patients begin exhibiting symptoms.
Hope for local dysarthric patients
“We can detect the decline of cognitive functioning through the patients’ daily use of language in communication.”
— Professor Helen Meng
Although databases for dysarthric patients are plentiful in the English language, no publicly available equivalent previously existed for Cantonese. The professors, together with a few other colleagues in CUHK, remedied this by carefully curating a selection of stimuli, the length of which range from single words to paragraphs and even full conversations in Cantonese. This allows them to gather data on the enunciation of certain words and vowels by dysarthric patients, as well as the rhythm of sentences they utter. As of 2021, some 27 dysarthric Cantonese speakers have participated in the project, generating up to 34 hours of data – a significant amount when compared to existing databases for other languages.
They have also been able to adjust their system so that it automatically translates dysarthric speech into normal speech, a predictive process on which Professor Meng has brought to bear her work in AI. Together, the two professors hope that their work will provide a foundation for further study into dysarthria in Hong Kong, which might, in turn, enable additional medical and therapeutic responses.
With their teams, Professors Wong and Meng have also done MRI experiments, with older adults at different cognitive levels as subjects, hoping to isolate the features of such decline by looking into how those affected might understand language or express themselves. “Ageing is a big risk factor in cognitive decline,” says Professor Meng. “We can detect the decline of cognitive functioning through the patients’ daily use of language in communication.”
Professor Wong adds, “Most studies look at this from an anatomical perspective: is this brain area thicker or thinner? Our research in addition examines the patterns of neural activities associated with performing a task.” These more “naturalistic” tasks enable the researchers to test for symptoms of cognitive decline, combining neuroscience and AI technology.
Professor Meng credits former Vice-Chancellor Charles K. Kao with establishing the Department of Electronics in the 1970s, which opened up CUHK’s research into electronic engineering and enabled it to get a foothold in the world of AI. “Artificial Intelligence is a newly added subject for ranking in the Best Global Universities Rankings 2023 by the US News & World Report, and it is encouraging that CUHK’s research on AI is ranked top in Hong Kong and third globally.” She adds that CUHK has also been very adept at bringing professionals across disciplines together, as their project has demonstrated.
A major problem the professors faced throughout their years of collaboration has been the relative lack of support for research involving the Cantonese language. While an estimated 80 million people speak Cantonese across the world, too often this is seen as insufficient (compared with, for example, Putonghua or English) to justify the allocation of significant research resources. Moreover, companies are reluctant to fund initiatives with limited commercial returns. Research in Cantonese speech, therefore, does not attract the same level of resources as more widely-used languages. Fortunately, the professors have received support from the Hong Kong Government’s General Research Fund, which has enabled the formulation of a Cantonese corpus for their work.
“Development across the lifespan”
Despite their long years of work together, the professors intend to continue their collaboration. For example, they are looking at how to support younger patients in language acquisition, and have developed an AI-based system that can accurately predict the progress of babies’ language skills. Professor Wong asks: “Can we develop more effective tools to help children and their parents or caregivers communicate with each other? Or perhaps we can help autistic children improve their social communication skills?” They see their series of projects not as attempts to fix isolated issues, but as part of a wider effort to overcome fundamental social problems arising from speech difficulties and disabilities.
