ReadAlong Studio: Application for Indigenous audiobooks and videos project

ReadAlong Studio is a web-based plug-in for Indigenous audiobooks developed by the NRC, Carleton University and Indigenous collaborators. Words on the screen are highlighted as they're read or sung aloud. The reader can also select any word to hear it spoken. The playback speed can also be slowed down to help students with learning the pronunciation.

• The software requires transcriptions of audio or video files and can be used with a wide range of languages.
• Educators can easily use the software with audiobooks their institution owns that contain speech or songs in an Indigenous language.
• The enhanced read-along versions of audiobooks created by the software are easy to use and effective for language learning.
• The technology has been applied to at least 27 Indigenous languages spoken in Canada, from several different language families.
• The software is open source, which means it's free and accessible to anyone in the world.
• People on 6 continents have made over 200 read-alongs in a variety of languages.

Underlying technologies

Two technologies power this software:

• Grapheme-to-phoneme (g2p) conversion, which converts a word (a sequence of graphemes) to its pronunciation (a series of phonemes)
• Matching the output of the g2p engine to the series of phonemes in the speech file

The speech models are trained on American English, but they do a good job of matching speech files from other languages with non-English phonemes.

Collaborators

• NRC Indigenous languages technology project
• Algonquian Dictionaries and Language Resources Project
• Carleton University School of Linguistics and Language Studies
• Northeastern University Khoury College of Computer Sciences

Ongoing and future work

Now that the software is being used worldwide, the team receives feedback and requests from users (sometimes by modifications to the code). There are 2 important improvements that are planned for the future:

• Adding the ability to edit alignment of the text with the speech so users can correct errors made by the software
• Simplifying the process for embedding software results into a website

Project team

• Fineen Davis, Programmer, NRC
• David Huggins-Daines, Independent Researcher
• Eric Joanis, Research Council Officer, NRC
• Patrick Littell, Research Associate, NRC
• Aidan Pine, Research Officer, NRC
• Shankhalika Srikanth, Student, NRC
• Mark Tessier, Programmer, NRC
• Delasie Torkornoo, Programmer, Carleton University
• Sabrina Yu, Student, NRC