Samuel Larkin

Roles and responsibilities

I am an Assistant Research Officer in the Multilingual Text Processing (MTP) team at NRC Digital Technologies (DT).

Current research and/or projects

* Using Common-Voice, create an web interface to record audio clips based on predefined sentences for our Indigenous Language Technologies project.

* Implement version 2 of YiSi, an automatic metric for evaluating machine translations.

* How to properly handle XML markup in Neural Machine Translation for a project with House of Commons Canada.

* How best to use fixed terms (terminology bank) during Neural Machine Translation.

Education

* MSc, in Statistical Machine Translation (Natural Language Processing) Université du Quebec en Outaouais 2004-2006

* BSc, computer science Université de Sherbrooke 1997-2000

Professional activities/interests

* Neural Machine Translation

* Neural Networks related to Natural Language Processing

* Curious in nature, I'm always up for a good challenge

Awards

* NRC IP Achievement Award (2021)

* NIST OpenMT 2012 1st place in translating Chinese-to-English 2nd place in translating Arabic-to-English

* 2014 TRAD 1st place in translating Arabic-to-French

* 2016 Conference on Machine Translation (WMT) 2nd place in translating news from Russian to English

* 2017 WMT:

* Tied for the 1st place in translating Russian-to-English 4th place in translating Chinese-to-English

* Sentence-level correlation on evaluating news translation 1st place in evaluating Chinese and Russian (from English) Tied for the 1st place in evaluating Czech, German, Finnish, Latvian, Turkish (from English) and 5 out of 7 English (from Czech, German, Latvian, Russian and Chinese) test sets i.e. No.1 in 12 out of 14 test sets

* Sentence-level correlation on evaluating medical text translation 1st place in evaluating all tested languages Czech, German, Polish and Romanian from English leading with a wide margin!

* System-level correlation on evaluating news translation 1st place in evaluating Latvian and Russian (from English)

* 2018 WMT:

* Sentence-level correlation on evaluating news translation Tied for the 1st place in evaluating all tested languages (from English) and 4 out of 7 English (from Czech, Russian, Turkish and Chinese) test sets i.e. No.1 in 11 out of 14 test sets

* System-level correlation on evaluating news translation 1st place in evaluating English-to-Russian and Turkish-to-and-from-English

* Corpus filtering, 4th place in 100 million filtered word evaluation 8th place in 10 million filtered word evaluation 6th place overall One of the only four submissions that achieved top 10 results in both evaluation settings (out of 48 submissions)

* 2019 WMT We participated in the Kazakh-Russian-English translation track. During this competition I heavily modified Sockeye (NMT) with a novel idea of using multiple sources to translate a source sentence. We ranked 4th in Kazakh-English and

* 2020 WMT I’ve participated in two low resource tracks, German-Upper Sorbian and Inuktitut–English. Our EN-IU system performed best out of the constrained systems in terms of BLEU. Our IU-EN system performed third-best out of all systems in terms of BLEU. We ranked second in Upper Sorbian-German and ranked third in German-Upper Sorbian.

I also help a team member with her Quality Estimation task.

* 2021 AmericasNLP We’ve participated in a low-resource translation task Spanish into Wixárika, Nahuatl, Rarámuri, and Guaraní. Our results consistently placed our submissions as the second-ranking team (behind Helsinki’s top 2-3 submissions) in the with-development-set group, and second or third ranking team (2nd, 3rd, or 4th submission) within the no-development-set cluster as measured by CHRF.

* 2021 WMT We participated in the Unsupervised MT and Very Low Resource Supervised MT, Lower Sorbian to and from German and Upper Sorbian to and from German, where we placed first or tied for first place. Quoting from the results “The most successful teams were NRC-CNRC, which was the best or on par with the best systems in all Sorbian tasks”