Have you ever struggled to learn a new language? Maybe you should spend less time trying to speak it and start singing instead!
This is the concept behind SLIONS, a new app being developed by Dr Ye Wang, an Associate Professor of Computer Science at NUS Computing. The app uses karaoke to teach foreign languages—an approach that some people may find more fun and engaging than vocabulary flash cards, writing exercises, or any of the other methods commonly used in language instruction.
The SLIONS Journey
“The motivation for this app came out of my own desire to speak better English,” says Prof Wang, a Chinese native who spent many years in Europe before joining NUS in 2002. “I started studying English at the relatively late age of 16, which meant that I had to work a lot harder than my peers who had early exposure to the language.”
Wang’s research indicated that singing might be a powerful tool for learning a new language. In 2012, he spent a semester as a Visiting Associate Professor at Harvard Medical School, where he worked with patients who had aphasia, a condition in which language ability is impaired due to a stroke or other brain injuries.
“We found that many patients could still remember how to sing even after they lost their ability to speak,” notes Wang. “This suggested that singing activates different parts of the brain than spoken language, opening alternate pathways to language acquisition and recall.”
That experience compelled Prof Wang, a globally-recognised expert in sound and music computing, to explore how new technologies can leverage the power of singing to improve language learning. His journey included many long days (and nights) in the lab, and a series of chance encounters that provided inspiration along the way.
One of those encounters came during a sabbatical in Canada in 2016, where he met Dr Doug Turnbull, an American computer scientist and kindred spirit who also specialises in music technology. Turnbull ended up becoming a senior visiting fellow in the Sound and Music Computing Lab at NUS, which Prof Wang directs, and played a key role in helping to conceptualise the SLIONS project.
Their work ultimately led to the first working prototype of the SLIONS app in 2018, which was developed with the support of two NUS students: Mr Riwu Wang, a recent graduate who served as the main app developer and won multiple awards for his work, and Ms Dania Murad, a master’s student who led the interaction design.
The SLIONS Experience
SLIONS, which stands for Singing and Listening to Improve our Natural Language, is a karaoke app that helps users learn songs in foreign languages. The app currently offers three genres of music—pop, classics, and kids. Most have simple melodies that will be familiar to most listeners, as well as clear lyrics that make them relatively easy to learn.
Once a song is selected, users can listen to an audio clip or watch a video of a professional singer performing it. The videos also show the translated text of the lyrics on the screen as they are being sung, which helps users to understand both the sound and the meaning of the words. They can then practice the song in “sing along” mode (with accompanying professional vocals) or “sing solo” mode (with only the backing track).
Then comes the really fun part. Once you think you have the song down, you can go to evaluation mode and test how well you actually know it. After singing it through, the app will provide you with a score relating to how accurate you pronounced the words in the song. You will get an overall score and a score for individual lines within the song. This helps you identify lines with a high share of misspoken words and practice them until you achieve mastery.
But what about those of us with a good ear for language but a tin ear for music? Don’t worry—the app only generates scores based on pronunciation and vocabulary, not pitch or style. As long as the words as spoken correctly and in time, you can still get a high score.
A Global Opportunity
After building the first prototype of the SLIONS app last year, Prof Wang and colleagues ran a pilot study with Chinese and English language students to assess its efficacy. Initial results were exciting. After trying the app, the students not only showed improved pronunciation and vocabulary, but also expressed satisfaction with overall user experience.
The study generated a global buzz among researchers focused on music and sound computing. It was accepted for presentation at the premier Association for Computing Machinery Multimedia Conference in South Korea last October.
Prof Wang now aims to expand on this research by testing the app with primary school students, who he believes will also find the experience engaging and derive even greater benefits than adults. He also envisions further collaborations with other NUS departments like the Centre for English Language Communication and the Centre for Language Studies, both of which provided helpful input on his work during the pilot.
“Singapore is the perfect place to develop an app like this,” says Wang. “It’s a multi-lingual society that is also open to new educational technologies.”
Prof Wang acknowledges that the app may not work for everybody and emphasises that it is intended to augment existing language learning programs, not replace them. But he also sees potential to scale this project globally, and perhaps even spin it into a commercial product.
“There are roughly 1 billion English language learners globally,” notes Prof Wang. “If we get this right, the opportunity for an app like this could be enormous.”
Paper:
SLIONS: A Karaoke Application to Enhance Foreign Language Learning
