ReadAlong Studio: Application for Indigenous audiobooks and videos project

Interactive story telling embeddable into any website!

Make your own interactive "read-along" and "sing-along" audiobooks for language learning with ReadAlong Studio. As the audio plays, the word currently being spoken is highlighted, and you can click on any word to hear it pronounced in isolation. The playback speed can also be slowed down to help learn the pronunciation.

Developed for Indigenous audiobooks by the NRC with Carleton University and Indigenous collaborators, this web-based plug-in is now available in a wide range of languages. Indigenous educators can easily use the software with any existing audio or video file and a transcription.

So far, users across the globe have made ReadAlongs and the technology has been applied to at least 27 Indigenous languages spoken in Canada, from several different language families.

The software is open source, which means it's free and accessible to anyone in the world, while respecting privacy and data sovereignty.

Underlying technologies

Two technologies power this software:

• Grapheme-to-phoneme (g2p) conversion, which converts a word (a sequence of graphemes) to its pronunciation (a series of phonemes)
• Matching the output of the g2p engine to the series of phonemes in the speech file with SoundSwallower

The speech models are trained on American English, but they do a good job of matching speech files from other languages with non-English phonemes.

Collaborators

• NRC Indigenous languages technology project
• Algonquian Dictionaries and Language Resources Project
• Carleton University School of Linguistics and Language Studies

Recent improvements

Now that the software is being used worldwide, the team receives feedback and requests from users (sometimes by modifications to the code). There are 2 important improvements that are planned for the future:

• Adding the ability to edit alignment of the text with the speech so users can correct errors made by the software
• Simplifying the process for embedding software results into a website

Project team:

• David Huggins-Daines, Independent Researcher
• Eric Joanis, Research Council Officer, NRC
• Patrick Littell, Research Officer, NRC
• Aidan Pine, Research Officer, NRC
• Marc Tessier, Programmer, NRC
• Delasie Torkornoo, Programmer, NRC (formerly Carleton University)

Past team members:

• Fineen Davis, Programmer, NRC
• Shankhalika Srikanth, Student, NRC
• Sabrina Yu, Student, NRC