Evaluation of the Digital Technologies Research Centre 2025–26

Table of contents

• Initialisms and acronyms
• Introduction
• Profile
• Scientific excellence
• Business innovation
• Capacity, competencies and facilities
• Progress toward identified priorities: Addressing government digital priorities
• Progress toward identified priorities: Enabling inclusive innovation
• Recommendations and management response and action plan
• Appendices
  • Appendix A: Methodology
  • Appendix B: Limitations and mitigation
  • Appendix C: Peer review committee members
  • Appendix D: Collaboration centres

Initialisms and acronyms

AI

Artificial intelligence

CSPS

Canada School of Public Service

DAC

Data Analytics Center

DT

Digital Technologies Research Centre

FWCI

Field-weighted citation index

GBA Plus

Gender-based Analysis Plus

HPC

High performance computing

ILT

Indigenous languages technology

IP

Intellectual property

MATS

Multimedia Analytic Tools for Security

NRC

National Research Council of Canada

OAE

Office of Audit and Evaluation

OFRM

Office of Facility Renewal Management

OGD

Other government department

PRC

Peer review committee

SGILE

Speech generation for Indigenous language education

SME

Small and medium-sized enterprises

TRL

Technology readiness level

VAM

Volumetric additive manufacturing

Introduction

An evaluation of the National Research Council of Canada's (NRC) Digital Technologies Research Centre (DT) was conducted in fiscal years 2023–24 and 2024–25 and covers the period from 2017–18 to 2023–24. The research centre was last evaluated in 2018, covering the period of 2012–13 to 2016–17. This evaluation was carried out in accordance with the NRC's approved evaluation plan and the Treasury Board's Policy on Results (2016).

This report begins with a profile of the research centre, followed by findings on scientific excellence, business innovation, capacity, competencies and facilities. It concludes with recommendations for improvement.

Evaluation approach

Approach

This evaluation was led by the NRC's Office of Audit and Evaluation. It applied a mixed- methods approach, incorporating both quantitative and qualitative data from several lines of evidence. A Gender-based Analysis Plus (GBA Plus) lens was applied where appropriate in the evaluation.

Methods

• Document review
• Data analysis
• Case study
• Bibliometric analysis
• Internal and external interviews
• Expert peer review

Evaluation questions

• To what extent is DT advancing scientific excellence in digital technologies within Canada and globally?
• To what extent is DT supporting digital technology-enabled business innovation including technology adoption and commercialization among private sector, academic and government clients?
• To what extent does DT have the capacity, competencies and facilities (e.g., high performance computers and physical facilities) to meet its objectives?
• To what extent has DT made progress towards its identified priorities (i.e., addressing government digital priorities and enabling inclusive innovation)?

Indigenous languages technology project case study

As part of this evaluation, an impact case study was conducted to provide an in-depth examination of the verb conjugator and text-to-speech projects within DT's Indigenous languages technology (ILT) project. Highlights are presented in the findings section of this report.

Background

From December 2017 to March 2024, Indigenous partners and DT researchers collaborated on 2 projects aimed at revitalizing Indigenous languages, verb conjugators and text-to-speech. Both projects emphasized a community-led collaborative approach, Indigenous data ownership and long-term sustainability through open-source frameworks.

Verb conjugator

Indigenous languages often lack resources due to a limited number of fluent speakers and the complex verb structures. In 2017, DT researchers began collaborating with an Indigenous educator to develop a digital verb conjugator. This tool aimed to generate and display verb conjugations, offering students an interactive and efficient way to learn verb formation.

Text-to-speech

In 2022, verb conjugator users identified a need to incorporate pronunciation assistance into the verb conjugators. In response, the speech generation for Indigenous language education (SGILE) project was formed to integrate text-to-speech systems into the verb conjugators.

Profile

The Digital Technologies Research Centre (DT) partners with diverse clients, collaborators and other NRC research centres to drive innovation and promote the adoption of digital technologies across various sectors. The research centre strives to maintain a balance between fundamental research and innovation while supporting a variety of initiatives that span the technology readiness level (TRL) scale.

DT has 10 initiatives that facilitate the delivery of its research, including 2 Challenge programs, 2 Cluster Support programs and leadership of 3 collaboration centres.

Overview

Vision and mission (2024 to 2029)

DT aims to contribute to a stronger and sustainable economy for Canada through digital technology research excellence and the responsible and ethical use of leading-edge digital technologies.

The research centre's mission is to lead innovation in digital technologies and drive their application and adoption across sectors, in collaboration with the Canadian government, businesses, academia, researchers and society.

Key strategic priorities

• Advance research excellence in artificial intelligence (AI), computer vision, natural language processing and quantum technologies through an expanding Canadian and global network
• Support business innovation by increasing opportunities for small and medium-sized enterprises (SMEs), facilitating research for commercialization and promoting innovation through NRC programs
• Address government priorities by informing policy decisions and supporting other government departments in adopting digital technologies
• Enable inclusive innovation by prioritizing equity, diversity and inclusion, supporting official languages, career development and employee wellness

DT has 10 initiatives that serve as the delivery mechanism for its innovative research:

• Digital Privacy and Security program
• Multimedia Analytic Tools for Security (MATS) program
• Data Analytics Center (DAC)
  • AI Accelerator for the Government of Canada
• Indigenous languages technology (ILT) project
• Atlantic Science Enterprise Centre (in development)
• Cluster Support programs:
  • Artificial Intelligence for Logistics
  • Digital Health and Geospatial Analytics
• Challenge programs:
  • Applied Quantum Computing
  • Artificial Intelligence for Design

In addition, DT supports the Action Plan for Official Languages 2023-2028.

Structure

Figure 1 – Text version

Hierarchical graph outlining the organizational structure of the NRC's Digital Technologies Research Centre. At the top is the Chief Digital Research Officer, overseeing 4 roles:

• Director General
• Executive Advisor and AI for Design Lead
• Generative AI Innovation Lead
• Director, Business Development

Reporting to the Executive Director are 4 directors and program leaders:

Director, Research and Development

• Computer vision and graphics
• Scientific data mining
• Text analytics
• Multilingual text processing

Director, Special Initiatives

• Strategic planning
• Capital investment processes
• Partnerships with science-based and R&D organizations

Director, Research and Development

• Crypto and quantum computing
• Cybersecurity
• Human–computer interaction
• Data science for complex systems
• Data analytics centre

Director, Operations

• Administration
• Project management support

Program leaders

• AI for logistics
• Multimedia analytic tools for security
• Digital health and geospatial analytics
• Advanced quantum computing

Clients and collaborators

Over the evaluation period, DT signed a total of 350 agreements with 85 unique clients and collaborators. These partners included Canadian and international universities, Canadian federal government departments, foreign governments, industry and others such as provincial governments, hospitals and industry associations.

Academia

• 31 clients and collaborators (29 Canadian and 2 international)
• 209 agreements^{Footnote 1}
• 98% Canadian and 2% international
• Not responsible for any revenue

Other government departments

• 21 clients and collaborators (20 Canadian and 1 international)
• 101 agreements
• total net revenue is $26.4 million^{Footnote 2}
• Responsible for 92% of DT's total revenue^{Footnote 3}

Industry

• 33 clients and collaborators (29 Canadian and 4 international)
• 40 agreements
• Total net revenue is $1.1 million^{Footnote 2}
• Responsible for 3% of DT's total revenue^{Footnote 3}

Human resources

Workforce

As of March 31, 2024, DT had a total of 183 staff^{Footnote 4}, with the largest proportion being research officers (60%).

• 108 research officers
• 27 computer science specialists
• 19 students
• 11 administration and corporate^{Footnote 5}
• 7 management
• 3 technical officers

DT's human resources are located across the country with a high concentration in Ottawa. In FY 2023-24, DT staff were distributed across 8 Canadian cities:

• Ottawa: 60%
• Fredericton: 11%
• Montreal: 9%
• Toronto: 5%
• Waterloo: 4%
• Moncton: 3%
• Victoria: 2%
• Edmonton: 2%

Financial resources

Revenues

DT generated $28.7 million in revenue over the evaluation period. Other government departments (OGDs) represented the largest source of DT's revenue across fiscal years, ranging from a low of 84% in 2018-19 to a high of 97% in 2022-23.

Technical services generated 58% ($16.6 million) of DT's revenues, while strategic research and development (R&D) contributed 42% ($12.1 million).

Expenses

Salaries, including employee benefits payments (EBP), represent the majority of DT's expenses (86%) and are the primary drivers of rising costs.

DT expenses over the evaluation period

Facilities

Digital facilities

A significant portion of DT's work is conducted nationwide using high-performance computing (HPC) facilities. The research centre primarily relies on its own HPC system, Beatrix, which is equipped with graphical processing units (GPU). Additionally, DT utilizes cloud computing and government resources through Shared Services Canada, which are equipped with central processing units (CPU).

Data Analytics Centre (Ottawa)

The Data Analytics Centre (DAC) collaborates with private and public sector organizations to analyse big data by combining expertise in AI, data science and machine learning with advanced algorithms and powerful computing clusters. The facility includes a secure, temperature-controlled computation room with HPC equipment, a visualization room with modern interactive dashboards and a room equipped with a model train for AI demonstrations.

Imaging and graphics facility (Ottawa, Montreal)

The imaging and graphics facility supports research in 3D imaging, including system design, data analytics, machine learning, big data visualization and data-driven simulation. The facility consists of a large laboratory area for longer range systems and human activity tracking research, a vision-driven cobotics and industrial imaging lab, a visualization room and an enhanced secret level security room for special projects.

Other facilities

Other facilities include the human‑computer interaction lab at the NRC's Montreal Road campus in Ottawa and the cybersecurity lab in Fredericton.

DT leads 3 collaboration centres (see Appendix D):

• CIC–NRC Cybersecurity Collaboration Consortium at the University of New Brunswick in Fredericton
• NRC–Fields Mathematical Sciences Collaboration Centre at the University of Toronto
• NRC–Waterloo Collaboration on Artificial Intelligence, Internet of Things, and Cybersecurity at the University of Waterloo

Scientific excellence

The Digital Technologies Research Centre (DT) collaborates on numerous projects and publications with NRC research centres, clients and partners across a broad range of research areas. These collaborations have contributed to a steady increase in the number of publications produced by its researchers each year. DT's average proportional citation impact has exceeded that of both the NRC and Canada overall, with particularly strong results in text analytics and scientific data mining.

The research centre's activities have made significant impacts both in Canada and globally, with the potential for future contributions in sectors such as health, defence, infrastructure and transportation. DT researchers have been publicly recognized for their contributions to advancing knowledge and technology, as evidenced by awards, conference invitations and committee participation. Clients and collaborators view DT as a leader in cutting-edge research in Canada.

Internal scientific collaboration

Collaborations with other NRC research centres focus on developing technologies and tools, often through knowledge transfer. For example, DT cybersecurity experts advised another NRC research centre on developing more secure and adaptable technology. DT is providing, or scheduled to provide, expert advice and leadership to 16 research-related initiatives across 10 research centres, as well as 2 corporate initiatives.

Many of the collaborations aim to leverage DT's expertise in AI and data analytics. Over the evaluation period, DT worked with 10 of the NRC's 14 research centres on a total of 30 scientific projects. The number of projects with NRC research centres increased steadily from 2 in fiscal year 2019-20 to 9 in fiscal year 2023-24.

The peer review committee (PRC) indicated that DT may be working in too many areas and should focus on the most relevant fields that have the greatest impact for Canadians. The committee recommended that the research centre develop mechanisms to ensure it is advancing the most relevant research areas.

DT had a total of 178 co-publications with other NRC research centres over the evaluation period, representing 23% of their total publications.

The annual number of the Digital Technologies Research Centre's co-publications with other NRC research centres increased from 2018 to 2023

DT has co-published with 13 unique research centres. NRC initiatives such as Challenge programs and Small Teams initiatives have increased funding and opportunities for cross-NRC collaborations, leading to more joint publications.

External scientific collaboration

Finding

DT's strong commitment to collaborative research is evident through numerous collaborative research agreements and co-publications across diverse research areas.

DT signed 208 collaborative research agreements, including 195 with Canadian organizations and 13 with international organizations. The majority (92%) were with academia. The number of agreements varied yearly, with a large number in 2020, 2021 and 2022, partly due to an influx of collaborative research agreements stemming from the NRC's Challenge programs.

DT's co-authorship rates are similar to those of the NRC. During the evaluation period, DT published 776 papers, 79% of which were co-published (compared to 82% for the NRC). Of these co-publications, 59% were with Canadian researchers (matching the NRC) and 38% were with international researchers (NRC 49%^{Footnote 6}).

The highest proportion of co-publications was with academia (73%), followed by industry (11%) and other Canadian government departments (5%). These figures closely mirror the NRC's rates of 75%, 13% and 6% respectively.

The focus of collaborations varies by client type:

• Collaborations with other government departments (OGDs) focus on applied solutions to enhance operational efficiency and support cost savings
• Academic collaborations aim to advance the understanding and principles of digital technologies through fundamental scientific investigations
• Industry collaborations focus on advancing technology by developing and refining tools and methods to achieve innovation goals

Scientific publications

Finding

DT has demonstrated scientific excellence through significant growth in its research publications in all identified research areas, including in top-tier conferences.

Scientific publication growth

Over the evaluation period, DT had a total of 776 publications. The number of publications increased from 2018 to 2023, representing an average growth of 24% per year, significantly higher than the NRC's overall rate of 2%. This growth was evident in all research areas and can be attributed to a rise in the number of researchers at DT working with universities through collaboration centres and challenge programs, as well as enhanced research with clients.

Figure 2 – Text version

Line graph showing the number of publications produced annually by the NRC's Digital Technologies Research Centre from 2018 to 2023, with a clear upward trend over the years.

Publication year	Number of publications
2018	61
2019	104
2020	121
2021	151
2022	160
2023	179

Publication venues

Publications were primarily conference papers, which is expected in computer science, accounting for 55% (compared to 65% for the NRC), followed by journals at 38% (matching the NRC), and books or chapters at 3% (NRC 2%). The percentage of conference papers likely reflects the role of conferences as a primary venue for presenting the latest research in computer science.

DT has published in top-tier conferences, including the International Conference on Learning Representations, the Annual Meeting of the Association for Computational Linguistics and the Conference on Neural Information Processing Systems.

Publication impact

Finding

DT's average citation impact exceeds that of both the NRC and Canada overall. Notably, text analytics and scientific data mining have particularly strong citation impacts.

Citation impact scores indicate the extent to which DT's publications are cited and used by other researchers worldwide to advance science. From 2018 to 2023, DT had a higher average field-weighted citation index (FWCI) (1.48) than both the NRC (1.32) and Canada (1.46). This can be attributed to the prominence of researchers within the research centre, as well as the significant expansion of, and interest in, AI in recent years.

However, DT had a lower percentage of publications (5.9%) ranking within the top 5% of most cited publications in Scopus compared to the NRC (7.5%). Text analytics and scientific data mining research areas have the highest share, with 10% of their publications ranking among the top 5% of most cited publications.

Digital Technologies Research Centre FWCI scores across research teams

Research impact

Finding

DT's research activities have made a significant impact in Canada and globally, with examples in health, defence, infrastructure and transportation highlighting the research centre's potential for future contributions. These efforts address real-world challenges with innovative solutions.

Some examples of research activities that have made or have the potential to make an impact both in Canada and globally include:

• DT supported the Aging in Place Challenge program by developing innovations in cybersecurity, privacy, health, activity monitoring, and age-tech adoption. For example, DT collaborated on sensor technology development to enhance the safety and well-being of older people through smart home innovations.
• DT created tools to analyze media sources in low-resource languages^{Footnote 8}, improving public safety and security. One project, for instance, created platforms for global surveillance of disease outbreaks and public security risks.
• DT researchers are developing advanced infrastructure monitoring methods, integrating satellite radar and in-situ sensors for bridge strain assessment to reduce physical and financial risks.
• DT researchers developed a solution to more efficiently respond to air traveler complaints. The tool uses AI to assess, classify and extract information from complaints, streamlining the process and reducing the need for multiple reviews. It facilitates better monitoring of compliance, ensuring that transportation providers adhere to regulations.
• Next-generation 911 services will soon allow people to send texts, photos and videos to provide valuable information about their situation. DT researchers are collaborating to ensure that emergency services can benefit from these advancements.

Scientific leadership

Finding

DT researchers demonstrate scientific leadership through public recognition of their contributions to knowledge advancement and technology development. This is evidenced by awards, conference invitations and committee participation. Clients and collaborators view DT as a leader in cutting-edge research within their focused research areas in Canada.

Awards

Between 2019 and 2024, DT researchers received a total of 15 external awards, most notably for best papers. In 2019, a DT researcher was recognized as a top writer for science by Medium, an online publishing platform. This accolade distinguished them among 50 writers, out of approximately 63,000 stories in the science category.

In 2022, another DT researcher received the prestigious Canadian Academy of Engineering Fellowship, known for its rigorous nomination and selection process. The PRC recognized a DT researcher's Best Paper Honorable Mention at the 2023 Annual Meeting of the Association of Computational Linguistics as a significant achievement.

Committees

Over the evaluation period, DT staff participated in approximately 110 committees. These include committees for conferences, symposiums, workshops, program or strategy reviews, and professional associations. Additionally, DT staff served as reviewers or editors for approximately 23 peer-reviewed journals.

Peer recognition

Most external interviewees perceive DT as a leader in cutting-edge research in Canada, and approximately half view them as leading internationally.

"DT would benefit from developing a formal system to track and present scientific leadership through awards, honors and high‑impact publications."

Conference engagement

Finding

Between 2018 and 2024, DT demonstrated scientific leadership by delivering presentations at approximately 276 conferences across 36 countries worldwide.

Virtual

DT presented at 101 virtual conferences, with the majority taking place in 2020 and 2021 due to the COVID‑19 pandemic.

North America

• DT presented at 51 conferences and attended 14 conferences in Canada
• DT presented at 35 conferences in the United States and at 1 conference in Mexico

South America

• DT presented at 4 conferences in Brazil

Europe

• DT presented at 40 conferences across 14 different countries in Europe, including France, Germany, Sweden, Denmark and United Kingdom

Africa

• DT presented at 5 conferences across 4 African countries, including Côte d'Ivoire, Morocco, Tunisia and Rwanda

Asia

• DT presented at 19 conferences across 9 different countries in Asia, including India, Japan, South Korea, Singapore and Thailand

Australia

• DT presented at 3 conferences in Australia

Business innovation

The Digital Technologies Research Centre (DT) assists external clients and collaborators in achieving their business objectives by developing new products, services and capabilities, improving operational efficiency, and offering training for highly qualified personnel. The research centre is advancing new technology, as evidenced by intellectual property (IP) and projects funded through grants and contributions. Although commercialization is a new priority for DT, initiatives and newly secured license agreements demonstrate progress in this area.

DT's client-focused approach has led to high client satisfaction. While the research centre has faced some challenges in engaging other NRC research centres, new strategies are being developed to address this issue. DT relies on specific government departments for revenue and recognizes the need to diversify its income streams and expand into the private sector.

Contributions to business innovation

Finding

DT assists external clients and collaborators in achieving their business objectives by developing new products, services and capabilities, improving operational efficiency and offering training for highly qualified personnel. The research centre's work is also cited in granted patents, demonstrating that its research is leading to the development of new inventions by Canadian businesses.

Benefits to clients and collaborators

Examples of how DT contributes to new products and services for Canadian organizations:

• In collaboration with the Security and Disruptive Technologies Research Centre, DT is developing 3D tomographic printing solutions for volumetric additive manufacturing. This has led to a family of patents across both research centres and has initiated licensing discussions with a Canadian start-up.
• Through the Artificial Intelligence (AI) for Design Challenge program, DT collaborated with an academic partner to develop AI models for designing photonic devices. The research centre is working with a start-up to develop a commercialization and licensing model for this innovation.

The peer review committee (PRC) acknowledged that DT contributes to job creation and economic growth in the digital technology sector through industry and government partnerships and support for start-ups and small and medium-sized enterprises (SMEs). DT also enhances innovation capabilities by providing clients and collaborators access to cutting-edge technologies, expertise and facilities.

Patent citations

During the evaluation period, 34 DT publications were cited by 56 patent families. Patent citations highlight the significant impact and value of DT's research by demonstrating how their work influences new innovations. Patent families, which group patents filed in multiple countries for the same invention, emphasize the global reach of DT's contributions to technological advancements.

The number of patent families citing DT publications varied across the fiscal years, with a low of 4 in 2019 and a high of 19 in 2023.

Technology advancement

Finding

DT is fulfilling its commitment to advancing new technology, as evidenced by the progression of IP and projects through the technology readiness levels (TRLs).

The progression of invention disclosures, patent applications and patents issued demonstrates that DT is advancing new technology through the TRLs, from early-stage concepts to market-ready technologies.

The Digital Technologies Research Centre's intellectual property technology readiness levels (fiscal year 2018-19 to 2023-24)

The PRC noted examples that illustrate DT's commitment to advancing digital technology research through the TRLs:

• DT has developed a novel approach to volumetric additive manufacturing, which enables faster printing speeds and the creation of complex geometries. This technology has progressed from initial research stages (TRL 1-3) to application exploration and commercialization trials (TRL 4-6).
• DT's work on chip design uses machine learning to predict and correct for fabrication imperfections. This technology has been successfully tested and is being further developed in collaboration with academic and industry partners (advancement towards TRL 4-7).
• DT's drone detection technology has advanced from lab-based exploration of event-based imaging cameras to military and police-tested operational prototypes. These prototypes outperform market leaders in their specific use cases and have led to commercialization discussions.

Focused approach increased patent applications

DT's patent applications increased over the evaluation period. The research centre attributes this increase to a stronger focus on commercially aligned innovations and efforts to maximize the efficiency of the patenting process.

Figure 3 – Text version

Bar chart showing the number of patent applications filed annually by the NRC's Digital Technologies Research Centre from fiscal year 2018–19 to 2023–24.

Fiscal year	Number of patent applications filed
2018-19	0
2019-20	1
2020-21	8
2021-22	9
2022-23	21
2023-24	29

Commercialization

Finding

DT has increased its prioritization of commercialization in its 2024-29 strategic plan and has identified supporting strategies to advance in this area. Although commercialization is a new priority for the research centre, initiatives and newly secured license agreements demonstrate progress in this area.

Strategic shift toward commercialization

DT's 2024-29 strategic plan prioritizes commercialization, with a focus on increasing engagement to boost the adoption and commercialization of products and services. There are plans to develop and implement commercialization strategies in 3 key areas of DT innovation, including AI photonics, volumetric additive manufacturing and bioinformatics.

The research centre has identified 2 pathways to commercialization and technology adoption:

• Facilitating the creation of start-up companies (for example, a series of AI photonics projects resulted in the formation of a start-up company in Canada)
• Removing obstacles for SMEs

Additionally, the PRC highlighted the Data Analytics Centre (DAC) as important in helping SMEs adopt AI and machine learning through data integration, modeling and prediction services.

Commercialization initiatives and achievements

Examples of DT's support for commercialization include:

• identifying technology areas with commercially-ready IP for testing and further research, such as machine vision drone inspection detection, 3D tomographic printing and AI in photonics design
• establishing IP committee guidelines to provide direction on protection and commercialization strategies among researchers and stakeholders

The research centre secured 4 new license agreements during the evaluation period, indicating active commercialization efforts. These licenses were for software or datasets created by DT to improve AI training. DT also works with the Industrial Research Assistance Program to attract SMEs and provides subsidies for client research work completed.

Engagement: Revenue and client diversity

Finding

DT is meeting revenue targets through collaborations with other government departments (OGDs). However, there is a recognized need to diversify revenue streams and expand its reach within the private sector.

Revenues increased significantly between fiscal years 2022-23 and 2023-24. In response to NRC-wide corporate budget cuts^{Footnote *} and rising operational costs for software licences and high-performance computing (HPC) hardware renewal, DT prioritized revenue growth through DAC offerings and collaborations with OGDs.

Digital Technologies Research Centre's revenue targets and actuals (fiscal year 2019-20 to 2023-24)

DT generates revenue through initiatives like DAC, the AI Accelerator for the Government of Canada and the Multimedia Analytic Tools for Security (MATS) program. The PRC noted that these revenues support DT's financial sustainability and enable further investments in research and innovation. To ensure long-term financial sustainability, the PRC recommended the research centre continue exploring ways to diversify its revenue sources.

Figure 4 – Text version

Pie chart showing the distribution of revenue sources for the NRC's Digital Technologies Research Centre:

• 92% from other government departments (OGDs)
• 4% from other sources
• 3% from industry

Approximately 40% of OGD revenue attributed to a single department.

While industry client revenues remained stable during the evaluation period, DT increased collaborations with SMEs, which remains a strategic priority for the research centre moving forward.

Engagement: Successes and challenges

Finding

DT has attained high levels of client satisfaction by prioritizing a client-focused approach. However, it has faced challenges in engaging research centres, which are being addressed through new strategies.

Strong collaboration outcomes

Clients and collaborators, both internal and external to the NRC, expressed high levels of satisfaction working with DT, with the majority rating their experience 10 out of 10. Key strengths highlighted include:

• DT's client-focused approach, demonstrated through a deep understanding of client needs, prompt practical advice and the rapid acquisition of expertise for project execution
• DT's professionalism, as shown in effective teamwork, flexible timelines, proactive communication and consistent leadership

The PRC praised DT for its services to clients and the value it provides.

"DT participates in conferences, workshops and other events to network with potential clients and collaborators. They also engage in outreach activities to raise awareness of their research and capabilities. They could further enhance their outreach and awareness efforts by actively participating in industry events, publishing success stories and targeted marketing campaigns."

Some challenges in research centre engagement

Challenges related to engagement include:

• DT has been managing high demand from other research centres while working with limited resources
• DT has not consistently been involved in the early planning stages of projects led by other research centers, which may have resulted in missed opportunities for revenue sharing. A new approach is being implemented to formalize collaborations, ensuring the research centre's early involvement and proportional revenue sharing.

Additionally, DT's advisory board identified opportunities to enhance client engagement, including promoting expertise in responsible AI, selecting projects with higher TRLs that benefit SMEs and forming larger, multidisciplinary project teams.

Capacity, competencies and facilities

To date, the Digital Technologies Research Centre (DT) has met client needs. However, as the field rapidly evolves, with shifting priorities and increasing demand from other NRC research centres, DT will need to prioritize its work moving forward. The research centre faces challenges in attracting and retaining top talent, but is implementing mitigation strategies to counter higher private-sector salaries, such as targeting early-career researchers and highlighting the advantages of working in the public service.

DT facilities, including high-performance computing (HPC), have been sufficient to meet collaborative research and client needs. However, concerns about the future have emerged due to aging infrastructure and the absence of a firm plan to upgrade its facilities. Enhancing HPC capacity is seen as critical to continue meeting the needs of collaborative research and clients.

Capacity

Finding

DT has had sufficient capacity to meet client needs to date. However, to address emerging demands in a rapidly evolving field with shifting priorities and increasing requests from other NRC research centres, DT will need to improve how it prioritizes its work.

High demand for services

DT faces challenges in advancing science, innovation and supporting partners due to limited resources and shifting priorities. Resources are stretched across multiple priorities, including Cluster Support programs, Challenge programs and contributions to various NRC initiatives.

DT's services are in high demand from other NRC research centres and corporate branches. Over the next 5 years, DT is expected to provide expertise and strategic advice to 10 research centres and 2 corporate branches for 16 initiatives.

Strategies to manage demand

Strategic priorities are constantly adapting to a fast-changing landscape. To manage capacity versus demand, DT uses strategies to balance available expertise and interest. DT prioritizes projects that are aligned with Canada's national priorities and focuses on projects that fit DT's strategic priorities.

DT does not accept all requests from NRC research centres. It prioritizes commitments to Challenge programs and revenue-generating programs and ensures alignment with DT researchers' interests and available expertise. Another strategy is to begin with a small part of a large request and consider future expansion.

The peer review committee (PRC) suggested that better prioritization is needed to maintain focus on key areas of strength.

Human resources capacity

Finding

DT faces challenges in attracting and retaining top talent in niche areas due to a highly competitive labour market and a lack of critical mass in certain research fields. To mitigate these challenges, the research centre is targeting graduate and post-graduate students and highlighting the unique advantages of public service.

Hiring and recruitment challenges

Challenges in attracting and retaining top talent include:

• a highly competitive labour market for digital expertise, which makes it difficult to attract talent in niche areas of quantum computing, AI, machine learning and natural language processing
• a lack of critical mass in certain research areas, which makes it challenging to recruit and retain experts who may not want to be the sole specialists in their field within an organization. Collaboration centres help address this issue by allowing individuals to work for the NRC while staying connected to the academic ecosystem.

These challenges have caused significant delays in hiring. While DT has successfully filled all positions with strong candidates, some hiring processes took much longer than expected.

Strategies to recruit and retain talent

DT targets graduate and postgraduate students to establish a strong talent pipeline. A large percentage of DT's hires have been students, ranging from 63% in fiscal year 2018-19 to 94% in 2020-21.

The research centre's hiring strategy emphasizes the unique benefits of public service and the NRC, such as contributing to the public good and offering a healthy work-life balance, to counter higher private-sector salaries. This approach has successfully attracted top talent, including 3 recently hired quantum computing experts. The PRC noted that DT's ability to attract high-calibre researchers and young professionals is a positive indicator of its strong reputation and appeal.

DT's retention strategies include career development opportunities, such as providing staff with time to write publications and attend conferences, as well as recognition programs. The PRC highlighted DT's commitment to mentoring early-career researchers, offering them opportunities to gain visibility by attending prestigious conferences and presenting at universities and workshops.

Facilities sustainability and operations

Finding

While DT's current facilities have met the needs of clients and staff, there are concerns about aging infrastructure and the absence of a firm plan for upgrades. Enhancing HPC capacity is seen as critical to continuing to meet the demands of collaborative research and client needs.

Alignment and use

DT's facilities align with the needs of small and medium-sized enterprises (SMEs), universities, and other government departments (OGDs). The research centre is working to expand facility usage among SMEs, focusing on the Data Analytics Centre's (DAC) artificial intelligence (AI) services.

Expanding infrastructure is essential

DT's strategic plan highlights the need to renew and expand HPC capacity to support both internal research and external collaborations. Adequate HPC capacity is critical for training AI models and essential to DT's success.

While current HPC capacity meets client needs, increasing capacity is vital for the future. The lack of recent capital investments for infrastructure updates presents challenges for equipment purchases and system maintenance. The PRC emphasized the need for a strategic plan to renew facilities, ensuring sustainability and effectiveness.

Recognizing limited resources, the DT advisory board recommended a targeted investment plan for AI and large-scale computing infrastructure. The NRC is examining future facility upgrades and investments as part of its departmental investment planning cycle.

Progress toward identified priorities: Addressing government digital priorities

The Digital Technologies Research Centre (DT) contributes to federal digital technology policy decisions by advising on federal artificial intelligence (AI) policies, influencing international standards, supporting new regulations and participating in collaborative projects. The research centre's publications are also cited in policy documents.

DT is recognized within the NRC as a leader in responsible AI development and use, focusing on addressing issues such as bias, explainability, inclusiveness and online safety. The research centre educates stakeholders on responsible AI through collaborative initiatives. Additionally, DT supports digital transformation within the NRC and across government by developing enabling technologies.

Policy impact

Finding

DT contributes to federal digital technology policy decisions by advising on AI policies, influencing international standards, supporting new regulations, participating in collaborative projects and producing publications cited in policy documents.

Advancing federal digital technology policies

DT supports the Canadian government by providing evidence-based information and ethical digital technology solutions for policy decisions. The research centre contributes to these decisions through the following activities:

• Supporting the Treasury Board Secretariat in AI policy: DT advises on federal AI policies, including the renewal of the Directive on Automated Decision-Making, the creation of content for the guide on the use of generative AI and the development of the government‑wide AI strategy.
• Defining international standards: DT's imaging and graphics facility enables the NRC to actively contribute to international standards development. This includes work with the American Society for Testing and Materials Committee E57 on 3D Imaging Systems, such as laser scanners and optical range cameras. Over the evaluation period, the facility contributed to the development of 9 international standards.
• Supporting new regulations: DT developed a proof-of-concept for analyzing public and business feedback on new regulations before they are enacted, ensuring these perspectives are considered in the regulation development process.
• Collaborative projects: DT's work in human‑computer interaction has influenced policy by establishing standards for autonomous vehicles, identifying risks and shaping related policies and regulations.
• Publications: DT's publications have been cited in policy documents in Canada and internationally. Over the evaluation period, 25 DT publications were cited in 10 policy documents from 9 policy bodies, including the Government of Canada, the AI Regulation Special Collection (France) and the European Parliamentary Research Service (Europe).

Responsible artificial intelligence development

Finding

Within the NRC, DT is recognized as a leader in developing and promoting ethical, responsible, trustworthy and secure AI. The research centre's contributions focus on key areas such as addressing bias, enhancing explainability, fostering inclusiveness and improving online safety.

Leadership role in responsible artificial intelligence

DT is conducting core AI research with responsible AI as a key focus. Its goal is to lead in the development of responsible AI solutions that safeguard digital information for Canadians and the Government of Canada. DT also aims to establish itself as a trusted partner for external clients and collaborators.

For instance, DT collaborated with internal partners to develop, test and pilot AI Zone, a tool designed for NRC employees. This secure alternative to consumer chatbots protects intellectual property (IP) and sensitive data.

Recognized as an NRC leader

DT is regarded by NRC researchers as a leading authority on AI, valued for its expertise in ethical and effective AI application.

Areas of focus

DT focuses on 4 key areas for responsible AI:

• Bias: DT researchers study biases and the propagation of stereotypes related to gender, race, age and social status, which includes assessing different sources of biased outputs.
• Explainability: DT integrates explainability into many applied AI projects, enabling humans to understand and interpret AI decision-making. This is essential for building trust, ensuring transparency and fostering effective human–AI collaboration.
• Inclusiveness: Projects prioritize inclusiveness by deploying technologies that support under-represented groups and communities, including Indigenous peoples.
• Online safety: DT focuses on detecting toxic language and identifying factors that contribute to suicide, aiming to enhance online safety.

The peer review committee (PRC) emphasized the need for a strategic focus in responsible AI, and recommended that DT identify specific areas where it can excel and collaborate most effectively.

Responsible artificial intelligence use

Finding

DT demonstrates leadership in responsible AI through collaborative projects and initiatives aimed at educating students, NRC staff, public servants and the public.

DT plans to train highly qualified personnel for the ethical and appropriate integration of AI into government and business activities. The research centre trains its students and employees on the responsible use of AI and has implemented internal processes to ensure all activities adhere to ethical AI obligations and principles.

Recent initiatives include:

• DT developed and launched a cross-NRC initiative focused on applying generative AI to address organizational challenges. As part of this effort, DT organized a comprehensive training on the responsible use of generative AI for summer students, with plans to extend this training across the NRC.
• DT held a session at the 2023 Artificial Intelligence for Logistics Program conference to discuss algorithmic assessment in responsible AI.
• In fall 2024, DT organized responsible AI training for NRC management in collaboration with the Canada School of Public Service (CSPS). Ongoing plans include sharing resources on responsible AI with other areas within the NRC.
• DT's management team and representatives from AI Accelerator for the Government of Canada regularly present at government and industry policy and outreach events, both in Canada and internationally.

Supporting federal digital transformation

Finding

DT contributes to digital transformation within the NRC and across the government by developing technologies that support digital transformation efforts.

Digital transformation at the NRC

DT is recognized as a key player within the NRC's data and digital ecosystem. Its Data Analytics Centre (DAC) collaborates with the NRC, industries and other government departments (OGDs) to develop AI prototypes, drive innovation and address emerging challenges.

Examples of DT's contributions to digital transformation initiatives within the NRC:

• The Chief Digital Research Officer is central to the NRC's digital transformation, contributing to the drafting of the digital transformation action plan, developing the data and digital strategy for the NRC's Knowledge, Information and Technology Services Branch, and collaborating with the data management team.
• DT's expertise and infrastructure support digital transformation across the NRC. For instance, DT improved the Client Service Centre's onboarding process by replacing spreadsheets with a system that imports data and generates predictive dashboards, streamlining daily operations.

Federal digital transformation

Examples of support to federal digital transformation initiatives include:

• DT partnered with Shared Services Canada to develop a cloud platform for data science and AI tailored to the federal public service. This platform enhances data analysis, reduces costs, improves decision‑making and protects sensitive information.
• DT developed automated dashboards to analyze the public service employee survey results, helping NRC managers address employee concerns more efficiently. These tools are expected to be adopted by other departments.

"DT is doing well in meeting the needs of other government departments and there is significant demand for its services."

Progress toward identified priorities: Enabling inclusive innovation

The Digital Technologies Research Centre (DT) integrates Gender-based Analysis Plus (GBA Plus) into its research and technology development, focusing on the needs of underrepresented populations. Equity, diversity and inclusion (EDI) are key priorities in the research centre's strategic plan. While women and Indigenous peoples are well-represented in DT's workforce, there is a lower representation of racialized persons and persons with disabilities.

Gender-based Analysis Plus

Finding

DT integrates GBA Plus considerations into its research and technology development, as evidenced by several ongoing projects aimed at supporting underrepresented populations.

Support for underrepresented groups

DT targets research benefiting underrepresented populations, such as Indigenous language revitalization, and aims for a geographically balanced client mix. While its early efforts focused on eastern and central Canada, DT is now expanding its engagement in western Canada, with specific actions planned for fiscal year 2024-25.

The research centre has connected with regional contacts through the Industrial Research Assistance Program (IRAP) and with Indigenous communities via the Indigenous languages technologies (ILT) project.

DT has several ongoing and upcoming projects addressing societal issues affecting underrepresented populations, though it is too early to assess their long-term impacts.

Notable examples include:

• Collaborating with Indigenous communities and language experts on Indigenous language revitalization. The peer review committee (PRC) recognized the ILT project as a strong example of inclusive innovation.
• Creating a tool that improves the management of transportation-related complaints and monitors compliance with accessibility standards, ensuring better service for persons with disabilities.
• Partnering with the University of Ottawa and collaborators, including the NRC's Human Health Therapeutics Research Centre, to establish the Advanced Institute for Dementia and Aging program and facility, which aims to provide clinical solutions for dementia.

The PRC noted that the impacts of DT's current efforts in inclusive innovation are unclear. The committee also recommended further clarification on how diversity and equity considerations are integrated into the design of research projects.

Workforce diversity

Finding

EDI is a priority in DT's new strategic plan. While women and Indigenous peoples are well-represented in the research centre's workforce compared to labour market availability, there is lower representation among racialized persons and persons with disabilities.

DT's 2024-29 strategic plan commits to attracting diverse candidates through various strategies. These include:

• highlighting the NRC's successes
• promoting women in science, technology, engineering and mathematics (STEM)
• providing EDI training
• advertising job opportunities to specific communities

By targeting candidates with diverse backgrounds and experiences, DT has reduced hiring challenges for Indigenous peoples and women.

Ratio of the Digital Technologies Research Centre's workforce made up of underrepresented groups relative to Canadian average labour market availability^{Footnote 9}

Recommendations and management response and action plan

Recommendations

Focusing on the most relevant research areas

The Digital Technologies Research Centre (DT) is involved in a large variety of projects across a broad range of research areas with internal and external clients and collaborators.

However, the research centre is currently engaged in too many projects across too many research areas. The peer review committee (PRC) suggested that DT develop a mechanism to prioritize research areas and projects that will be the most relevant in the future.

Given DT's position within an evolving and complex research ecosystem, the research centre should ensure it has a distinct research agenda that maximizes its strategic advantages and delivers the greatest impact for Canada.

Recommendation 1

DT should work with its advisory board to further define the research areas it will prioritize. It should also develop an approach to selecting the most relevant projects to pursue within these research areas.

Diversifying revenue streams

DT generates revenue through initiatives like the Data Analytics Centre (DAC), AI Accelerator for the Government of Canada and Multimedia Analytics Tools for Security (MATS) program. The PRC noted that these revenues contribute to DT's financial sustainability and support further investment in research and innovation.

However, to ensure long-term financial sustainability, the committee recommended that DT continue exploring strategies to diversify its funding sources.

Currently, the research centre's revenue largely comes from other government departments (OGDs), with approximately 40% attributed to a single OGD. DT recognizes the need to diversify its revenue streams and expand its reach within the private sector.

Recommendation 2

DT should develop and implement strategies to diversify its revenue streams, expanding its reach within both the private sector and other government departments.

Long-term renewal plan

The evaluation identified concerns regarding the research centre's future capacity due to aging infrastructure and the absence of a clear plan to upgrade and modernize DT facilities. The research centre's strategic plan acknowledges the need for enhancements to improve the quality and capacity of DT's work, noting that inadequate infrastructure could potentially impact research and recruitment efforts.

The PRC recommended developing a comprehensive facilities renewal plan to ensure the research centre has access to the state-of-the-art resources and infrastructure necessary for cutting-edge research. Additionally, the PRC noted that a strategic plan for facilities renewal would support long-term sustainability and effectiveness.

Recommendation 3

DT should develop a facilities renewal plan to identify and prioritize its present and future facilities' requirements and set out options for accessing the state-of-the-art infrastructure necessary for cutting-edge research. The plan should also identify facilities' requirements for upgrading and maintenance.

Management response and action plan

Recommendation 1

DT should work with its advisory board to further define the research areas it will prioritize. It should also develop an approach to selecting the most relevant projects to pursue within these research areas.

Risk-level: Moderate

Management response: Accepted

Action: DT will work with its advisory board over the next 2 to 3 meetings (in spring and fall 2025 and in spring 2026) to further define the research areas it will prioritize and develop an approach to selecting the most relevant projects to pursue within these research areas.

This will be accomplished by fine-tuning its current dynamic prioritization process in order to provide a more focused value moving forward. A more formal review of prioritisation within teams may also be implemented.

Measure of achievements:

1. The prioritization process and initial research areas are defined
2. An approach for re‑affirming the selection of relevant projects and initiatives is developed, documented and approved by DT's vice-president

Proposed person responsible: Director General, DT

Expected date of completion: October 31, 2026

Recommendation 2

DT should develop and implement strategies to diversify its revenue streams, expanding its reach within both the private sector and other government departments.

Risk-level: Moderate

Management response: Accepted

Action: DT will develop and implement strategies to diversify its revenue streams, and expand its reach within both the private sector and other government departments (OGDs), while building on the current success of the Data Analytics Centre (DAC), AI Accelerator and MATS programs which target small and medium-sized enterprises (SMEs) and OGDs.

These strategies will aim to diversify DT's engagements over revenues (e.g., more sectoral diversity will be targeted through the new AI for Productivity Challenge program and by accessing other research centres' networks with private sector companies). New revenue streams are possible as well (e.g., via the AI safety-related work in the context of the Canadian AI Safety Institute).

Measure of achievements:

1. A document of strategies to diversify revenue streams is developed
2. An indicator and target to diversify revenue streams is created

Proposed person responsible: Director General, DT

Expected date of completion: October 31, 2026

Recommendation 3

DT should develop a facilities renewal plan to identify and prioritize its present and future facilities' requirements and set out options for accessing the state-of-the-art infrastructure necessary for cutting-edge research. The plan should also identify facilities' requirements for upgrading and maintenance.

Risk-level: Moderate

Management response: Accepted

Action: DT will develop a facilities renewal plan, in consultation with other research centres. The plan will identify the facilities needed for its cutting-edge research and priorities, and set out options and plans for access, including through:

1. partnering with universities
2. seeking funding external to the research centre, including partnering with other organizations
3. promoting additional government support for sovereign AI compute investment

Measure of achievements:

1. The DT facilities renewal plan is created

Proposed person responsible: Director General, DT

Expected date of completion: October 31, 2026

Appendices

Appendix A: Methodology

Document review

Internal and external documents were reviewed to provide context and to complement other lines of evidence. These documents included strategic and operational plans, the previous evaluation, facility reviews, meeting minutes and others.

Data review

DT's administrative and performance data for 2018 to 2024 were reviewed to provide information on scientific excellence, business innovation, capacity and progress toward identified priorities. These included financial, human resources, project, client and intellectual property (IP) data.

Impact case study

An impact case study was conducted to provide an in-depth examination of the outcomes and impacts of the speech generation for Indigenous language education (SGILE) project within the Indigenous language technology (ILT) project. The case study examined the contributions of verb conjugators and text-to-speech projects to language revitalization and inclusive innovation. It also explored their broader implications for Indigenous communities and the global knowledge base in natural language processing.

Bibliometric study

The NRC's intelligence and analytics team conducted a bibliometric analysis of the Digital Technologies Research Centre (DT) for 2018 to 2024. DT's publication dataset was compiled by searching the Scopus database for all NRC-affiliated publications. Relevant publications were then identified based on references to DT in the metadata or authorship. The research centre validated the list and categorized the publications into 5 domains of expertise.

Key informant interviews

Interviews were conducted to collect information related to scientific excellence, business innovation, capacity and progress toward identified priorities. The information obtained through the interviews was used in conjunction with that of other lines of evidence.

A total of 39 stakeholders were interviewed for this evaluation:

• 14 DT interviewees (executives, directors, team and program leads, project-specific staff and advisory board members)
• 4 interviewees from other NRC research centres
• 21 clients and collaborators external to the NRC

Peer review committee

An international peer review committee (PRC) was convened in Ottawa from October 8 to 10, 2024, to assess DT's performance during the evaluation period. The committee consisted of 7 experts (see Appendix C) in key areas of digital technology.

The committee was provided with preliminary findings of the evaluation ahead of the site visit, along with other supporting documentation.

The site visit included a series of presentations by DT staff, question and answer periods and a tour of DT's facilities. Following the site visit, the PRC finalized a report, including recommendations, which was reviewed for factual accuracy by the research centre.

Appendix B: Limitations and mitigation

Below are minor methodological limitations encountered during this evaluation, along with the mitigation strategies.

Single case study conducted

A single case study was conducted to pilot an impact case study approach at the NRC. While a single case study can provide valuable in-depth insights, it is important to consider its limitations in terms of generalizability, potential bias and the inability to conduct comparative analyses.

Mitigation: To mitigate this limitation, case study findings did not form the basis for any evaluation findings. They were used to highlight and supplement existing information where relevant.

Incomplete data on scientific leadership

Awards, committee participation and conference engagements were not systematically tracked by the research centre, which may have led to underreporting of results in this area.

Mitigation: Gaps in the data are explicitly stated in the findings.

Use of citation impact to measure scientific excellence

The evaluation used field-weighted citation impact (FWCI) scores as a measure of scientific excellence. It should be noted that this analysis was conducted only on DT publications included in Scopus.

Additionally, the impact of more recent publications may continue to grow over time, as citations generally increase.

Mitigation: Other lines of evidence, such as data and document reviews and the PRC, were also used to assess scientific excellence.

Appendix C: Peer review committee members

Ozge Yeloglu (committee Chair)
Vice-President, Advanced Analytics and Artificial Intelligence
Canadian Imperial Bank of Commerce (Toronto, Canada)

Mallory Durran
Group Director, Data Science, Data Science Practice
Nesta (London, England)

Dr. Arvind Gupta
Professor, Department of Computer Science
University of Toronto (Toronto, Canada)

Dr. Alfredo Cuzzocrea
Professor of Computer Engineering
University of Calabria (Rende, Italy)
Director, Big Data Engineering and Analytics Laboratory (iDEA Lab)

Dr. Vanessa Thomas
Senior Advisor, Technology Review, Government of Canada (Ottawa, Canada)
Research Centre Advisory Board member

Manav Gupta
Chief Technology Officer
IBM Canada (Toronto, Canada)

Dr. Sanjeena Dang
Associate Professor, School of Mathematics and Statistics, Carleton University (Ottawa, Canada)
Canada Research Chair, Data Science and Analytics

Appendix D: Collaboration centres

CIC–NRC Cybersecurity Collaboration Consortium (CNCCC)

The CNCCC brings together researchers from the University of New Brunswick's Canadian Institute for Cybersecurity and DT to conduct research in cybersecurity.

Areas of focus include critical infrastructure protection, smart homes, enterprises, cities and smart grids.

The CNCCC's expertise includes:

1. network security
2. Internet of Things security
3. mobile security
4. malware analysis and attribution
5. dataset generation and distribution
6. machine learning
7. human behavior analytics
8. natural language processing
9. image and data analytics

NRC–Fields Mathematical Sciences Collaboration Centre

The NRC–Fields Mathematical Sciences Collaboration Centre comprises scientists from DT and the Fields Institute for Mathematical Sciences at the University of Toronto.

Examples of collaborations include projects that apply math and AI to health, energy and manufacturing, such as AI for precision discovery in biological systems, mathematical modeling of the SARS-CoV-2 lifecycle and COVID-19 vaccine response, quantum-enhanced design for materials and chemistry, and AI-driven process optimization in friction stir welding.

NRC–Waterloo Collaboration on Artificial Intelligence, Internet of Things, and Cybersecurity

The NRC–Waterloo Collaboration Centre on Artificial Intelligence, Internet of Things, and Cybersecurity unites researchers from the University of Waterloo's Faculty of Engineering and Faculty of Mathematics with DT.

Areas of focus include:

• battery-free touch sensors for the Internet of Things
• neuromorphics for vision-based movement planning and control
• reliable gesture recognition in virtual reality environments
• secure and scalable quantum-safe blockchain for critical infrastructure
• automated material discovery using deep reinforcement learning

Research expertise encompasses:

• AI
• the Internet of Things
• cybersecurity
• machine vision
• human-computer interaction