ReadAlong Studio: Application for Indigenous audiobooks and videos project

ReadAlong Studio is a web-based plug-in for Indigenous audiobooks developed by the NRC, Carleton University and Indigenous collaborators. Words on the screen are highlighted as they are read or sung out loud. The reader can click on any word to hear it spoken.

ReadAlong Studio requires existing transcriptions of audio or video files and can be applied to a broad range of languages. The technology is currently in use with Algonquin, Atikamekw, Northern East Cree, Southern East Cree, Gitksan, Gwich'in, Hän, Kaska, Mohawk, SENĆOŦEN, Seneca, Tagish, Upper Tanana, Tlingit, Northern Tutchone and Southern Tutchone. It could also be applied to Michif, Moose Cree, Swampy Cree, Inuktitut, Western Inuktitut, Kwak'wala, Mi'kmaq, Ojibwe, Tsuut'ina, and other languages.

Collaborators

• Algonquian Dictionaries and Language Resources Project, School of Linguistics and Language Studies, Carleton University
• Michael Running Wolf, Khoury College of Computer Sciences, Northeastern University
• NRC's Indigenous Languages Technology Project

Objectives

• Develop the underlying custom grapheme-to-phoneme (g2p) engine needed to power the alignment of audio and text
• Test the technology with Indigenous and non-Indigenous languages and for different types of texts (stories, songs, videos) to ensure its robustness
• Short-term: Align spoken words (audio or video) to existing transcripts (text) and produce videos of this
• Long-term: Design an easy-to-use interface where audio and text is visually aligned, and write simple code so that ReadAlong Studio can be easily embedded into any website. Our goal is to enable Indigenous communities to make their own ReadAlong videos
• Build a function to export to EPUB documents used by many e-readers; to Praat and ELAN files used in popular linguistic annotation software; and to standard subtitle formats for integration with videos and YouTube

Deliverables

• ReadAlong Studio Web App
• ReadAlong videos developed by the NRC, Carleton University and communities:
  • Complete:
    • Algonquin: 7 videos by the NRC team
    • Atikamekw: 10 videos by the Carleton U team
    • Northern East Cree: 4 videos by the Carleton U team
    • Southern East Cree: 3 videos by the Carleton U team
    • Seneca: 1 video by Seneca expert Robert Jimerson
  • In progress:
    • Gitksan: 3 videos by the NRC team
    • Gwich'in: 1 video by the NRC team
    • Hän: 1 video by the NRC team
    • Kaska: 1 video by the NRC team
    • Mohawk (Ohsweken dialect): 3 videos by the NRC team
    • SENĆOŦEN: 2 videos by the NRC team
    • Tagish: 1 video by the NRC team
    • Upper Tanana: 1 video by the NRC team
    • Tlingit: 1 video by the NRC team
    • Northern Tutchone: 1 video by the NRC team
    • Southern Tutchone: 1 video by the NRC team
  • Planned or possible:
    • Michif
    • Moose Cree
    • Swampy Cree
    • Inuktitut
    • Western Inuktitut
    • Kwak'wala
    • Mi'kmaq
    • Ojibwe
    • Tsuut'ina

Project team

• Delasie Torkornoo, Programmer, Carleton University
• David Huggins Daines, Volunteer
• Eric Joanis, Research Council Officer, NRC
• Aidan Pine, Research Officer, NRC
• Patrick Littell, Research Associate, NRC
• Mark Tessier, Programmer, NRC
• Fineen Davis, Programmer, NRC
• Shankhalika Srikanth, Student, NRC
• Sabrina Yu, Student, NRC