Evaluation of the National Research Council of Canada's international strategy 2024-25

Initialisms and acronyms

AIIR-Power

Artificial intelligence enhanced design and manufacturing of infrared photonic power converters for power and telecom

CIIP

Canadian International Innovation Program

CINUK

Canada–Inuit Nunangat–United Kingdom Arctic Research Programme

CiRA

Center for iPS Cell Research and Application

CRA

Collaborative research agreement

CSTIP

Collaborative Science, Technology and Innovation Program

FTE

Full-time equivalent

FWCI

Field-weighted citation index

G&Cs

Grants and contributions

ICAP

International Co-Innovation Action Program

IGC

International Governance Committee

IIO

International Innovation Office

iPS

Induced pluripotent stem

IRO

International Relations Office

MNE

Multinational enterprise

MOU

Memorandum of understanding

NPBS

National Programs and Business Services

NRC

National Research Council of Canada

NRC IRAP

National Research Council of Canada Industrial Research Assistance Program

R&D

Research and development

RTO

Research and technology organization

SME

Small and medium-sized enterprise

STI

Science, technology and innovation

UK

United Kingdom

VP

Vice-president

Introduction

The evaluation of the National Research Council of Canada's (NRC) international strategy, the first corporate strategy for international activities, was conducted in the fiscal years (FY) 2023-24 and 2024-25. The evaluation was carried out in accordance with the NRC's Departmental Evaluation Plan and the Treasury Board's Policy on Results (2016). This is the first evaluation of the NRC's international strategy, covering a 5-year period from its inception in FY 2019‑20 through to 2023-24.

This report begins by providing a profile of the strategy. It then presents findings on the extent to which the strategy achieved its 4 goals and was effectively implemented. Following the evaluation findings are recommendations for improvement.

Evaluation approach

Scope

The NRC's international strategy aims to advance the NRC's international engagement and collaboration through 4 goals. This evaluation assessed the extent to which the international strategy made progress on these goals, with a focus on 3 ecosystem economies (Germany, Japan and the United Kingdom [UK]).

International strategy goals

• Increase the NRC's international reputation as a global science, technology and innovation leader
• Advance knowledge and capabilities in key strategic areas
• Provide international access for Canadian small and medium-sized enterprises (SMEs)
• Implement effective governance

Approach

This evaluation was led by the NRC's Office of Audit and Evaluation. It applied a mixed-methods approach, incorporating both quantitative and qualitative data from several lines of evidence. This allowed for the triangulation of the data. A Gender-based Analysis Plus lens was applied where appropriate in the evaluation.

Methods

The evaluation included:

• document review (internal and external sources)
• data review (administrative, financial, performance and bibliometric)
• interviews with NRC staff (n=28) and external partners and stakeholders (n=21)
• case studies of 3 international projects (1 from each of the 3 economies)

More information can be found on the NRC's evaluation methodology in Appendix B, and on the methodological limitations and mitigation strategies in Appendix C.

Profile

The NRC's international strategy, the first corporate strategy for international activities, was launched in FY 2019-20 for a 5‑year period. It was then extended for 1 year through FY 2024-25 to ensure that an update of the strategy would be informed by this evaluation and the NRC's new strategic plan. The strategy aimed to focus the NRC's international activities by setting priorities and goals, aligning resources and tracking progress.

The implementation of the strategy resulted in the following changes for the NRC:

• increased focus on selected ecosystem economies
• establishment of an NRC presence in 2 of 3 ecosystem economies
• creation of the International Innovation Office (IIO) to support implementation of the strategy
• development of a governance structure with vice-president champions for each ecosystem economy
• allocation of funding from the Collaborative Science, Technology and Innovation Program (CSTIP) to NRC partners for research with ecosystem economies
• prioritization of supporting Canadian small and medium-sized enterprises (SMEs) collaborating with businesses in the ecosystem economies through the National Research Council of Canada Industrial Research Assistance Program (NRC IRAP) International

Evolution of the international strategy

International collaboration advances science, technology and innovation by leveraging resources, capabilities and knowledge to facilitate access to complementary resources, capabilities, networks and markets. The NRC's long history of international activities includes collaborative research with international partners, participation in multilateral engagements, involvement in international observatories and supporting Canadian SMEs in their efforts to enter global markets. The NRC, however, lacked a corporate strategy to focus international activities and did not have operational or grant funding for co-investment nor incentive mechanisms to encourage or enable collaboration between the NRC and international researchers.

Prior to FY 2019-20, the NRC had 2 units supporting international engagement, the International Relations Office (IRO) and NRC IRAP International. The IRO was responsible for advancing the NRC's international relations and establishing connections with complementary international partners. Meanwhile, NRC IRAP International supported Canadian SMEs in international co-innovation projects. These projects involve collaborations with international research and development (R&D) partners through various programs, including Eureka and the Canadian International Innovation Program (CIIP). The CIIP, a Global Affairs Canada program delivered with IRAP, targeted countries identified by Global Affairs Canada, including China, India, Brazil, South Korea and Israel, before the international strategy was implemented.

Other international research and technology organizations (RTOs), such as Germany's Fraunhofer-Gesellschaft and Australia's Commonwealth Scientific and Industrial Research Organisation, had established offices abroad to enhance international collaboration. Similarly, international innovation agencies supporting industrial R&D, like Business Finland and the Center for the Development of Industrial Technology in Spain, had also set up international offices before the NRC established its own international presence as part of its international strategy.

Origin

The NRC's international strategy was developed in response to the NRC Dialogue in FY 2016-17. The NRC Dialogue was an assessment of the state of the NRC, based on consultations with approximately 2,500 NRC staff. It emphasized the importance of international engagement and recommended that the NRC develop an international strategy that included:

• connections between research centres and international partners to achieve science excellence
• proposals for collaborative R&D programs
• NRC IRAP support for Canadian SMEs seeking to expand globally

Overview of the international strategy

The NRC's international strategy was launched in FY 2019-20 for a 5-year period, then extended for 1 year through FY 2024‑25. The strategy aimed to focus the NRC's international activities by setting priorities and goals, aligning resources and tracking progress. The strategy included 4 specific goals:

• International reputation (international relations): The NRC is recognized as a global science, technology and innovation leader and top of mind to engage for collaboration with the Canadian and global innovation ecosystem
• Advancing knowledge and capabilities (research excellence): The NRC's knowledge and capabilities in key strategic areas are advanced through attraction of leading international scientists, innovators and resources from ecosystem economies for strategic engagement in its collaboration programs
• Canadian SME access to international collaboration (business innovation): Canadian SMEs and groups of SMEs achieve accelerated entry into international markets and global value chains, enabled by international co-innovation and effective multinational enterprise (MNE) engagements, towards increased export and growth
• Governance: Effective mechanisms, tools and structures are in place to support coherence and implementation of the NRC's international strategy

COVID-19

The launch of the international strategy occurred shortly before the outbreak of the COVID-19 pandemic, which had serious impacts on its implementation. NRC staff and external collaborators felt that the COVID-19 pandemic significantly hindered international engagement due to lockdowns, border closures and travel restrictions. Virtual platforms and the NRC's new international presence helped sustain existing relationships, but were less effective for forming new connections. Many international projects were significantly delayed due to the need for travel (e.g., to enable access to specialized equipment or in-person collaboration).

Organizational structure^{Footnote 1}

The Vice-President (VP) of NRC IRAP and VP of Business and Professional Services share responsibility for implementation of the international strategy and co-chair the NRC's senior level committee for international engagement.

The National Programs and Business Services (NPBS) Branch manages and coordinates the NRC's corporate programs, client relationships and intellectual property. The branch includes 117 full-time equivalents (FTEs) (not all of whom support the international strategy).

Organizational roles^{Footnote 2}

The International Innovation Office (IIO), established in FY 2017-18, supports the framework, governance and monitoring of the international strategy. Its mandate includes focusing on international priorities, aiding in strategy implementation, communications and advice for decision-making. The IIO also participates in the governance of Eureka. As of July 2024, following the merger of the IIO with NPBS, the strategy and governance functions of the IIO are supported by a senior advisor. This advisor has access to resources from various directorates and the Director General's office as needed.

• Eureka is an international industrial R&D network with over 47 economies, including Canada. The NRC serves as Canada's National Office for Eureka, led by the President, working with the VP of NRC IRAP and Director General of NPBS.

The International Relations Office (IRO) offers strategic advice and support to research centres and executives to enhance international relations. It promotes the NRC as a preferred partner, supports targeted R&D collaborations that contribute to research excellence and innovation in Canada. The IRO manages the NRC's international office in Japan and its presence in Germany, oversees the Grants for International Affiliations program and ensures Canada's adherence to the International Science Council. The office is supported by 12 FTEs.

The NRC IRAP International supports business innovation by connecting Canadian SMEs with funding, advisory and innovation services. These initiatives are supported by 32 FTEs, including industrial technology advisors who identify international programs and opportunities for Canadian SMEs.

The National Program Office, established in FY  2017-18, coordinates and oversees the NRC's CSTIP and administers grants and contributions (G&Cs) to eligible external collaborators. The office is supported by 37 FTEs. While there is no dedicated full-time staff, the required resources are leveraged to deliver the international strategy.

• The Collaborative Science, Technology and Innovation Program (CSTIP) is a transfer payment program that provides $3 million per year in G&Cs for international collaborative research projects aligned with the international strategy. It began funding projects in FY 2019‑20.

The NRC's research centres leverage international contacts and connections to establish partnerships. They deliver on senior executive international commitments and collaborate with international partners to enhance capabilities, knowledge and networks.

Ecosystem economies

During the period of the international strategy, the NRC continued global collaborations in several countries based on strong cooperation in specific domains, while focusing on deepening relationships with 3 target economies, Germany, Japan and the UK. The goal was to work at an ecosystem level to build collaborative infrastructure, including broader networks, platforms and executive relationships, to enable more effective and impactful joint initiatives with shared funding. These countries were chosen for their strong science, research and advanced manufacturing sectors. Additionally, they share similar goals, compatible operating environments and alignment with the NRC and Canada's R&D.

Implementing the international strategy with an ecosystem economy required allocating resources to build the necessary infrastructure, including networks, platforms and processes. It also involved establishing relationships to align priorities and funding to facilitate collaboration. Utilizing best practices from other international research and technology organizations (RTOs) as part of the strategy, the NRC established an on-the-ground presence in both Germany and Japan, and further leveraged Global Affairs Canada in the UK to support the NRC's activities there.

• The NRC established a presence in Germany in December 2019, with an NRC employee with scientific credentials co-located at the Consulate of Canada in Munich to strengthen research linkages within the German ecosystem.
• The NRC opened its Japan office in Tokyo in October 2019. A former executive of a major Japanese company was contracted to build networks with potential science, technology and innovation (STI) partners and to open doors with Japanese industry.
• The NRC worked closely with an STI counsellor (Global Affairs Canada) at the Canadian High Commission in London to advance NRC goals in the UK.

A summary of key outcomes related to each ecosystem economy can be found in Appendix A.

Financial and human resources

Advancing the international strategy and other NRC international activities utilized resources from NPBS and NRC IRAP, totaling $146.1 million from FY 2019-20 to 2023-24. This included:

• $101.6 million in NRC IRAP International G&Cs awarded to Canadian SMEs, with 51% allocated to projects conducted with the international strategy's ecosystem economies
• $7.9 million in CSTIP G&Cs awarded to projects with researchers from ecosystem economies (minimal funding until 2021-22, as most CSTIP programs began that year)
• $3.3 million in G&Cs allocated to maintain Canada's membership in global science bodies under the Grants for International Affiliations program. Approximately 25% of this funding (FY 2019‑20 to 2021‑22) and 50% (FY 2022-23 to 2023‑24) was for association with Eureka.
• $29.2 million allocated for salaries and operations for NRC IRAP International, the IRO and the IIO
• $4.2 million for budget transfers, including $1.7 million for internal supports to research centres, branches and NRC IRAP, and $2.6 million to Global Affairs Canada for the NRC's Germany presence

International strategy spending increased from $23.5 million in FY 2019‑20 to $33.5 million in FY 2023-24, driven largely by G&Cs from CSTIP and NRC IRAP International, which grew from $17.6 million to $25.1 million.

Human resources per ecosystem economy

• Executive champion: An NRC VP assigned to develop relationships in an ecosystem economy and communicate and coordinate commitments within the NRC
• NRC IRAP International: 1 FTE per ecosystem economy to advance the business innovation goal
• IRO: 1 FTE senior advisor and 0.5 FTE program officers per ecosystem economy to advance the reputational goal and support the research excellence goal, plus broader office support (e.g., administration of NRC's participation in international bodies like the International Science Council)
• Overseas representatives: A senior advisor and an administrative staff person in Germany, and an NRC representative in Japan with part-time support staff

Goal 1: International relations

The following sections outline the evaluation findings related to each of the international strategy's 4 goals.

First, the strategy aims to increase the NRC's international reputation as a global science, technology and innovation leader. Executive leadership played a crucial role in fostering international engagement, with a significant increase in high visibility events and memorandums of understanding (MOUs). The NRC's participation in international networks like Eureka also raised its visibility and facilitated collaboration. Aligning executive commitments with research centres' capacity is crucial for the NRC's international reputation.

International relationships

In FY 2021-22, the NRC's International Innovation Office (IIO) assessed the NRC's relationships with the 3 ecosystem economies, placing each on a maturity scale.

• Level 1: Nascent
• Level 2: Emerging
• Level 3: Growing
• Level 4: Advanced
• Level 5: Mature

The relationship with Germany was assessed as "mature", since Germany had broad and deep linkages with the NRC. The UK was assessed as evolving to "mature", with broad research linkages with academic institutions, but business partnerships were limited to specific sectors. Finally, Japan was assessed as evolving from a more "nascent" relationship at the start of the strategy, as linkages with the NRC were deep but narrow, although growing broader with select partners.

As a result of the different levels of maturity, different tactics were effective in deepening collaboration. Documents and interviews indicated that:

• the NRC's Munich presence was crucial for supporting German partners in navigating NRC funding and deepening connections for German–Canadian collaborations
• the NRC provided funding and research support for collaborative projects as its relationship with UK organizations was maturing. Both NRC staff and external partners noted that the UK and the NRC share a similar approach to research and innovation, with aligned research priorities
• executive-level interactions, including minister-to-minister and president-to-president engagements, were key to deepening a relationship with Japan. NRC staff felt that the extensive experience of staff in the Tokyo office helped the NRC engage with Japanese organizations

Executive engagement

Direct engagement by NRC executives was often the first step in building institutional-level international relationships. Executives, such as the President of the NRC, vice-president (VP) champions or other VPs were involved in 70% of all engagements to advance international relations during the strategy.

According to the International Relations Office (IRO) database, executives represented the NRC at events and forums, made introductions, explored opportunities and helped maintain relationships. NRC staff reported that executive-level support was crucial for establishing international relationships.

External partners highlighted that executive-level engagement, in-person visits and NRC facility tours were important for initiating new relationships. These events allowed external collaborators to identify complementary capacities and shared priorities.

Examples of executive engagements

• In 2019, the President of the NRC attended the inaugural Research and Development 20 (RD20) for Clean Energy Technologies conference, an initiative of the 2019 G20 Osaka Summit in Japan, and also visited the Japanese Prime Minister at his residence.
• In 2021, the President of the NRC attended the 50th anniversary kick-off event for Germany–Canada science, technology and innovation (STI) cooperation with the German Minister of Education and Research and the Canadian Minister of Small Business, Export Promotion and International Trade.
• In 2022, the NRC's VP of Engineering, who also serves as the UK champion, delivered welcome remarks at the official launch of the Canada–Inuit Nunangat–United Kingdom Arctic Research Programme (CINUK) on the HMS Protector, a UK Royal Navy vessel, in Quebec City.

International relations events

According to the IRO's tracking database, the number of events to advance international relations increased during the strategy period, averaging 35 events per quarter in FY 2023-24, up from 24 in FY 2021-22. These events, held in Canada and abroad, included:

• high-level meetings with leaders from foreign academia, business, government and research and technology organizations (RTOs)
• joint committees, roundtables and workshops to plan collaborations
• conferences and other scientific fora

Of these, 57% were with Germany, Japan or the UK and tended to involve ministerial and executive-level engagements with high visibility compared to those with other economies.

NRC staff and external partners stated that international MOUs with RTOs and funding agencies generally provide access to ecosystem economies, fostering joint programming and collaboration. Active MOUs grew from 6 in FY 2018-19 to 11 by FY 2023-24, with the most significant growth in Japan, likely due to the emerging relationship there.

Increase in active memorandums of understanding for ecosystem economies

New NRC MOUs were signed with:

• the National Institute of Advanced Industrial Science and Technology of Japan to enable Aging in Place Challenge program projects
• UK Research and Innovation and Inuit Tapiriit Kanatami to deliver Arctic and Northern Challenge program projects
• Germany's Federal Ministry of Education and Research and the Natural Sciences and Engineering Research Council of Canada to support new 3+2 projects

International networks and forums

Canada joined Eureka as an associate member in 2012 and became a full member in 2022. Canada was one of the first non-European countries (with South Korea) to be admitted. For FY 2024-25, the NRC co-chairs the Eureka network with Germany.

IIO manages Canada's Eureka National Office with operational delivery by NRC IRAP International. The NRC engages in multilateral meetings related to Eureka governance and program delivery. Eureka provides NRC IRAP International clients, researchers, large enterprises and academics with a first point of contact and access to the expansive global network.

According to external collaborators, the NRC's participation in this multinational forum has provided the NRC with access to an extensive international R&D collaboration network and raised its visibility among Eureka members.

NRC staff and external collaborators identified leadership roles in international bodies and forums as critical to building visibility for the NRC and developing relationships. For example, the NRC participated in the Funding Agency Presidents' Meeting, the Science and Technology in Society forum, co-chaired the Research Institutes Leaders forum, and co-founded and co-chaired the Research Technology Organizations' International Network.

Some NRC staff noted that the NRC could further leverage its international networks such as Eureka to enable partners to access funding to support collaborative efforts. It was also noted that the NRC's recent membership in Horizon Europe may offer additional funding opportunities for research centres in the future.

The NRC's international visibility

Most external collaborators noted an improvement in the NRC's visibility during the period of the international strategy. German, Japanese and UK partners valued the NRC's international presence and close connection to their respective economies, as it demonstrated commitment and facilitating cooperation.

Most NRC staff also indicated that recent international activities enhanced the NRC's visibility, as shown by more committee and event invitations, alignment with ecosystem economies' funding cycles and more formal agreements.

Fulfilling commitments establishes the NRC as a reliable collaborator, fostering long-term international relationships. NRC staff identified the main reputational risk for the NRC with respect to international activities as the potential difficulty for research centres or programs to fulfill executives' international commitments. This risk stems from limited operational funding, competing research priorities and hesitancy towards international collaboration. Mitigation involves raising awareness of funding opportunities, aligning international opportunities with research goals and addressing hesitancy. The significant time needed to negotiate complex agreements with foreign partners and reduced federal travel budgets were also noted as potential risks.

• The Japan Science and Technology Agency (JST) and the NRC signed a Memorandum of Cooperation for research collaboration on October 3, 2022 at the Kyoto International Conference Center.

Goal 2: Research excellence

The strategy aims to advance knowledge and capabilities in key strategic areas.

The international strategy advanced research excellence through increased international collaborative research agreements (CRAs) and co-publications with ecosystem economies. International collaborations enabled access to global funding, facilities and expertise, thereby enhancing research capabilities and training highly qualified personnel. Projects that showcased significant advancements in research excellence included:

• the artificial intelligence enhanced design and manufacturing of infrared photonic power converters for power and telecom (AIIR-Power)
• the Japan trilateral projects between the NRC, Concordia University and Kyoto University's Center for iPS Cell Research and Application (CiRA)
• the Canada–Inuit Nunangat–United Kingdom Arctic Research Programme (CINUK)

International collaborative research agreements

Finding

The NRC increased its participation in international CRAs during the strategy period. This reached a peak of 193 CRAs by FY 2022-23, more than doubling from the baseline of 91 CRAs. The total value of these CRAs quadrupled, rising from $7.4 million in FY 2018-19 to a peak of $35.3 million in FY 2022-23, before decreasing to $30.7 million in FY 2023-24. The increase in new international CRAs since the implementation of the strategy was partly due to Collaborative Science, Technology and Innovation Program (CSTIP) projects.

Figure 2 includes ongoing international agreements that began in FY 2015-16 or later. The total annual value was estimated by dividing each CRA's total value by its duration in fiscal years (e.g., a 4-year, $1 million CRA counts as $250,000 per year). CRA amendments were included to account for additional project spending.

The proportion of new international agreements that were CRAs increased during the period of the international strategy. Prior to the strategy (FY 2018-19), CRAs made up 6% of all new international agreements compared to 10% during the strategy period. This represents a shift in the NRC's international activity towards more substantial and longer-term research over more transactional international agreements (e.g., research testing or technical services).

A greater number of the NRC's new CRAs were with Germany, Japan and the UK, indicating that the strategy's focus on these ecosystem economies led to increased collaborative research in those regions. Ecosystem economies were included in 20% of new international agreements (up from 15% in FY 2018-19) and 43% of new CRAs (up from 25% in FY 2018-19).

Notable international CRA trends include:

• The average value of new CRAs with Germany increased from a baseline of $0.5 million (FY 2018-19) to an average of $1.3 million.
• The NRC signed, on average, 5 new CRAs per year with Japan, up from a baseline of 1 (FY 2018-19).
• For the UK, the proportion of international agreements that were CRAs increased from 5% in 2018-19 to 23% during the strategy period.

Key international projects

The Japan trilateral projects (FY 2022-23 to 2025‑26) focus on accelerating induced pluripotent stem (iPS) cell therapies for cancer and other diseases using engineered cells from a patient's own body. According to NRC staff and external partners these projects have led to:

• new relationships among Canadian and Japanese researchers
• enhanced knowledge in the use of AI for gene editing predictions
• increased NRC awareness and new collaborations with the Center for iPS Cell Research and Application (CiRA)
• training of Canadian students

The AIIR-Power project (FY 2021-22 to 2023-24) aimed to develop artificial intelligence techniques to design and manufacture elevated-voltage photonic power converters. According to NRC staff and German partners, this research project has:

• improved relationships among NRC and German researchers, with the potential for further future collaborations
• led to enhanced solar cell design with advanced techniques
• validated models used in software development
• enhanced knowledge in the use of AI for the development of complex optoelectronics, which has implications for the energy and telecommunication sectors

The CINUK (FY 2021-22 to 2024-25) included 13 projects with Inuit involvement. These projects examined climate-driven changes to the terrestrial, coastal and near-shore marine environments in Inuit Nunangat, as well as the impacts on Inuit and community health and well-being.

Project partners included Inuit Tapiriit Kanatami, United Kingdom Research and Innovation, Polar Knowledge Canada, Parks Canada and the Fonds de recherche du Québec. According to NRC staff and partners, the projects led to:

• operational maps for Inuit communities to avoid ice casualties
• sharing of information on reducing carbon emissions in northern communities
• enhanced satellite technology
• greater engagement and collaboration with Inuit communities
• shared understanding of the barriers and opportunities of Inuit Nunangat

Co-publications and field-weighted citation impact

Bibliometric analysis found that the proportion of the NRC's co-publications that included authors from the 3 ecosystem economies remained relatively stable, at 18.3% pre-strategy (FY 2014-2015 to 2018-2019) and 19.0% during the strategy period (FY 2019-2020 to 2023-2024). It should be noted that there is a time lag between collaborations and co-publications, therefore co-publications may not immediately reflect recent collaborative efforts.

The highest proportional increase can be seen for Japan, which rose from co-publishing 4.5% of NRC's publications to 5.0% (a 11.1% growth), while the UK went from 10.0% to 11.0% (a 10.0% growth) and Germany went from 10.0% to 10.6% (a 6.0% growth). The higher pre-strategy proportions of co-publications with Germany and the UK may be due to their proficiency in English, well-established relationships and easy access to academic institutions.

The FWCI for NRC's co-publications with researchers from the 3 ecosystem economies was 2.1, exceeding the overall NRC FWCI of 1.3. This trend existed before the implementation of the strategy. The FWCI of NRC co-publications with Japan decreased from 2.2 pre-strategy (FY  2014-2015 to 2018-2019) to 1.7 during the strategy (FY 2019-2020 to 2023-2024).

Alignment and benefits for Canadian researchers

The 2022-23 NRC operational plan template was updated to include a section for international collaboration intentions. In FY 2022-23 and 2023-24, all research centres included activities with at least 1 international partner from the 3 ecosystem economies, aligning with strategic priorities. Activities included 3+2 projects, Challenge programs and international partnerships.

As joint funding through Challenge programs is thematically focused, the funding will naturally align more closely with the objectives of some research centres than with others.  As a result, some research centres seeking access to international funding found it difficult to line up their capabilities and research directions with the international funding opportunities through the calls for proposals.

Additionally, some research centres, like the NRC's Herzberg Astronomy and Astrophysics Research Centre and the Metrology Research Centre, work on a multilateral basis due to the nature of their research and are not directly engaged with the ecosystem economies.

Under the strategy, dedicated funding from CSTIP and NRC IRAP International for ecosystem economies benefitted Canadian researchers. From the International Innovation Office (IIO)'s 2023 Return on Investment report, it was estimated that an NRC investment of $13.6 million for the 3+2 projects leveraged over $78.6 million in project value. This complements findings from a 2020 co-innovation study, which estimated that IRAP International's investment leveraged additional investments from collaborators, amounting to as much as 6 times the total project value.

Leveraging international partnerships

Case studies highlighted that the NRC leveraged facilities, technologies and expertise from international partners.

• AIIR-Power project: The NRC accessed Europe's largest solar research and development (R&D) institute
• Japan trilateral projects: NRC researchers leveraged gene-editing technologies from CiRA in Japan
• CINUK projects: NRC researchers gained international expertise on renewable energy and polar ice conditions

NRC staff and external collaborators reported that international collaborations combined expertise and facilities for strategic research. These collaborations provided access to unique capabilities, fostering knowledge exchange and training highly skilled personnel.

Funding was identified by NRC staff and external researchers as a key factor for facilitating knowledge advancement and international collaboration.

NRC staff and external partners recognized that strong partnerships among ecosystem economies was a key enabler of knowledge advancement in international collaborations.

In Germany, the country's robust infrastructure and governance for R&D support (including various funding structures and consistent funding with predictable increases), were highlighted as effective enablers of knowledge advancement.

Under-served populations

Gender inclusivity and diversity

Most NRC staff and some external collaborators reported that they considered gender inclusivity and diversity in their international programs, projects and staff activities. Some programs and projects were specifically designed with inclusivity and diversity in mind, such as:

• the CINUK projects, which had a strong focus on Inuit self-determination in research. For example, they are guided by an MOU that ensures that any research includes and benefits the communities of Inuit Nunangat)
• the Aging in Place Challenge program projects with Japan that focus on improving the quality of life of older adults and their caregivers through a sustainable model for long-term care that shifts the focus toward preventive home and community-based care

NRC staff considered equity diversity and inclusion in the design of teams for Challenge program projects, 3+2 projects, other co‑innovation program projects and delegations.

Goal 3: Business innovation

The strategy aims to provide international access for Canadian small and medium-sized enterprises (SMEs).

The international strategy contributed to business innovation by increasing international funding and project opportunities for Canadian SMEs through NRC IRAP International. The number of new international projects per year nearly doubled, facilitating technological advancements and international market access for Canadian SMEs. These projects led to significant returns on investment, attracted substantial external funding and drove revenue and employment growth.

Increased funding opportunities

At the outset of the strategy, an internal study by the International Innovation Office (IIO) confirmed that NRC IRAP International provided Canadian SMEs with an effective model to access global value chains through co-innovation projects. The report concluded that these international co-innovation projects:

• helped Canadian SMEs grow
• enabled client access to international organizations
• facilitated durable partnerships
• leveraged NRC IRAP investments

Funding for NRC IRAP International clients rose by 176%, from $9 million per year before the strategy to $24 million per year during the strategy.

As shown in Figure 4, the share of funding for ecosystem economies increased from 34% to 42%, with significant growth in projects with Germany, the UK and Japan.

International projects for Canadian industry

As a result of the strategy, Canadian SMEs accessed more ecosystem economies and international markets. The annual average of new NRC IRAP International projects rose from 54 pre-strategy (FY 2014-15 to 2018-19) to 131 during the strategy (FY 2019-20 to 2023-24), with the share of projects in ecosystem economies rising from 26% to 30%. Specifically, Germany averaged 19 projects annually (up from 11), Japan 14 (up from 0) and the UK 14 (up from 5).

Among these international projects were International Co-Innovation Action Program (ICAP) projects, which started in the first 2 years of the strategy. ICAP helps Canadian SMEs analyze potential international collaborations, develop action plans and benefit from co-innovation programs and co-funding support. A total of 30 ICAP projects were completed, 40% in Germany, 40% in the UK and 20% in Japan.

Beyond ICAP projects, industrial technology advisors supporting NRC IRAP International facilitated partnership development activities and trade missions to connect Canadian SMEs to global value chains. In FY 2020-21, 12 events with 145 SME participants surpassed the target of 100. In Japan, 50 business-to-business meetings and the scouting of 130 SMEs led to 11 projects between Canadian SMEs and Japanese MNEs, exceeding the target of 8.

As previously mentioned, the IIO's 2023 Return on Investment report showed that NRC's investment in the 3+2 projects and NRC IRAP International's contributions to co-innovation projects significantly leveraged external funding. These investments advanced the strategy's business innovation and research excellence goals.

Interviews, data analysis and case studies indicate that international co-innovation projects led to invention disclosures and technological advancements with commercial potential and spin-off projects. Examples include artificial intelligence (AI) simulation tools and construction technology designed for northern climates.

Both NRC staff and external collaborators agreed that NRC IRAP International was instrumental in providing Canadian SMEs with access to international opportunities including matchmaking facilitated by industrial technology advisors, partnership development activities and 3+2 projects. Many external collaborators described NRC IRAP International as essential for their business growth.

Revenue and employment growth

From NRC IRAP projects and final reports, international clients experienced an average of 42% growth in revenue and an average of 25% growth in employment during the strategy (FY 2019-20 to 2023-24).

Companies supported by NRC IRAP International experienced slightly higher revenue growth compared to those supported by NRC IRAP before the strategy was implemented (29% versus 27%). They also saw more significant annual growth (42% versus 33%), except in FY 2023-24. Additionally, these companies maintained higher employment growth both before and during the strategy, indicating that the strategy contributed to their enhanced performance. It is important to note that stronger companies are more likely to receive funding for international co-innovation activities.

Goal 4: Governance

The strategy aims to implement effective governance.

The International Governance Committee (IGC) provided a platform for discussion and an advisory function to the NRC President on international activities. Overall, the IGC was seen as effective in its oversight role and in promoting a shared agenda for the NRC. Challenges were noted regarding the frequency of meetings and the potential for a lack of cohesive messaging at the executive level.

Despite significant organizational changes during the strategy's implementation, roles and responsibilities related to international activities within the NRC were generally viewed as clear.

The International Governance Committee

"The IGC is responsible for supporting the NRC International Innovation Office in managing and delivering the NRC's international engagement, including leadership and strategic direction for international priority setting. The committee reviews and recommends for approval various activities which support and enable the NRC's overall international engagement context."

The IGC is co-chaired by the Vice-President (VP) of NRC IRAP and the VP of Business and Professional Services, with active participation from 4 research division VPs. VP champions were assigned to Germany, Japan and the UK, while another VP oversaw both Taiwan and Korea.

IGC meetings discussed performance measurement, international activities in research centre operational plans, ecosystem and emerging economies, mission debriefs, upcoming events and memberships in international bodies. In the later years of the strategy, meetings occurred less frequently than the 4 times per year stipulated in the terms of reference. The International Innovation Office (IIO) identified contributing factors, including scheduling challenges, IIO restructuring and reduced capabilities, and numerous VP changes.

Committee meetings from fiscal year 2019-20 to 2023-24

Document reviews and NRC staff indicate that the IGC effectively advised the President, providing oversight and a platform for discussing a shared international agenda.

While the IGC serves in an advisory capacity, decision-making authority resides with the President. Opinions among IGC members varied regarding whether the IGC should have greater decision-making power. Although the IGC provides general oversight and advice, each VP is responsible for briefing the President on their respective areas. Some members noted that this could potentially lead to a lack of cohesive messaging to the President regarding international direction.

Changes to the organizational structure

Accountability for the strategy goals at the NRC is shared by 2 divisions. The International Relations Office (IRO) handles international relations and research excellence, while NRC IRAP International focuses on business innovation. There were changes to the organizational structure and engagement fora over the period of the strategy.

Initially the strategy was managed by the Secretary General (responsible for the IIO) and the VP of NRC IRAP. Over the period, there were significant changes to senior leadership.

In 2020, the international director general working group replaced the ecosystem economies working groups to coordinate strategy implementation with the research centres, NBPS and the IIO.

In 2022, based on input from committee members, the International Governance Committee (IGC) recommended that engagement with the research centres should be done through the broader Research Executive Director and Director General Committee. This decision eliminated the separate international director general working group and reducing the number of NRC committees.

Also in 2022, the IIO and IRO merged with NPBS to better coordinate research centre partner and client relationships and to leverage the expertise in grants and contributions (G&Cs) management from CSTIP and NRC IRAP International. The responsibilities of the Executive Director of the IIO were divided between the Director of NRC IRAP International and the new director general of NPBS. NPBS was renamed National Programs and Business Services International Innovation Office (NPBS IIO) to reflect the joint focus with the merger.

NRC staff generally found roles and responsibilities related to the strategy clear. Some thought that resources did not match the high ambitions of the strategy causing pressure on the IIO to deliver the strategy, while supporting executive requirements and broader coordination with Government of Canada stakeholders.

Implementation

The implementation of the NRC's international strategy led to more coordinated and aligned international activities, especially with ecosystem economies. Global circumstances such as geopolitical tensions both helped and hindered the strategy's implementation. Communication challenges within the NRC were reported. The NRC's International Innovation Office (IIO) demonstrated both the competency and capacity to support the strategy's goals and ecosystem economies.

Initiatives to enhance awareness and alignment

In 2021, an external assessment of NRC research centre engagement in supporting delivery of the NRC's international strategy was conducted 2 years into the strategy's implementation. It identified challenges hindering NRC researchers' participation in international collaborations. These challenges included understanding the strategy, communication, costs, available resources and contracting difficulties.

In response to the external assessment, the IIO intensified efforts to enhance awareness, alignment and coherence across the NRC. Initiatives included:

• implementing the A.C.E. model in 2021 for the International Relations Office (IRO) to support research centre communications and engagement
• conducting international research collaboration development workshops for research centres in 2022
• outreach to research centres to clarify the strategy and identify international opportunities

According to NRC staff, these initiatives and the strategy's focus on ecosystem economies have brought about a cultural shift within the NRC.

NRC staff noted that with the implementation of the strategy, more research centres and programs are viewing international collaboration as a means to access and leverage new capabilities and deliver on their priorities.

Observed changes in alignment

External collaborators observed significant improvements in the NRC's international engagement, including more joint planning, increased calls for proposals and better alignment with their priorities.

Most NRC staff noted a stronger focus on ecosystem economies and enhanced coordination with other Canadian government departments, such as Innovation, Science and Economic Development Canada and Global Affairs Canada, to align Canadian interests. However, a few staff members did not notice any significant changes or felt that changes were inconsistent across the NRC. This is to be expected as the strategy was not intended to equally impact all parts of the organization.

Both NRC staff and external collaborators viewed Canada as an attractive partner due to its climate change commitments and low security threats. Geopolitical events like the Russian invasion of Ukraine, changes with Canada's relationship with China, and the UK's exit from the European Union have heightened interest and engagement from ecosystem economies.

Competency and capacity

NRC staff reported that NPBS IIO has the competencies and capacity to support the strategy's goals and activities. They have been successful, especially considering this was the NRC's first corporate international strategy. However, additional resources would be needed if more ecosystem economies were added or scope of efforts otherwise expanded.

NRC staff emphasized the crucial role of vice-presidents (VPs) in communicating international priorities and commitments to research centres and programs. However, both the 2021 assessment and NRC staff noted that these priorities were not consistently and clearly communicated across the NRC.

Although information on international partnerships and the IRO, including calls for proposals, missions and success stories, was available on the intranet, employees noted the lack of a concise document that clearly explained the NRC's international strategy. This made it difficult for NRC staff to easily access and understand this information.

Recommendations and Management response and action plan

Supporting rationale and recommendations

The NRC's international strategy achieved its goals and provides an effective model for the NRC's international engagement and collaboration. However, there are a few opportunities to improve the strategy going forward.

Strategy document

Throughout the evaluation, it was challenging to clearly and easily understand the scope and components of the NRC's international strategy. In addition, while general information on international partnerships and the International Relations Office (IRO) was available on the NRC intranet, there was no concise document that clearly explained the NRC's international strategy.

Governance

NRC staff indicated that the International Governance Committee (IGC) was effective in providing oversight and acted as a platform for discussing a shared international agenda for the NRC. However, from FY 2021-22 to 2023-24 the IGC met less than the 4 times a year set out in the terms of reference.

Furthermore, opinions varied on whether the IGC should have more decision-making authority and some members noted a potential for inconsistent messaging at the executive level.

Management response and action plan

Recommendation 1

Refresh the NRC's international strategy documentation. The strategy should clearly outline the components and provide a specific and clear plan of:

• how goals will be achieved
• specific measures and expected milestones
• how progress will be tracked
• timelines
• resources
• roles and responsibilities

This revised strategy should be communicated widely across the organization for awareness, and to inform research centre and program planning, where relevant.

Risk-level: Low

Recommendation 2

Review the IGC to identify potential improvements, such as implementing consolidated high-level reporting informed by insights and recommendations from all IGC members, along with detailed reporting from the responsible VPs.

Risk-level: Low

Appendices

Appendix A: Key implementation outcomes

Germany

International relations

• The NRC's relationship with Germany is considered mature, with Germany being a strong international partner with extensive linkages to the NRC.
• NRC staff and partners reported that the NRC Munich presence strengthened ties with Germany and provided essential support for navigating NRC funding.
• Active memorandums of understanding (MOUs) increased from 4 to 5 over the period of the strategy (FY 2018-19 to 2023-24).

Research excellence

• New international collaborative research agreement (CRAs) with Germany averaged $1.3 million in value, up from a baseline of $0.5 million in FY 2018-19, due to larger multilateral agreements.
• The NRC collaborated with Germany on its Materials for Clean Fuels, Advanced Manufacturing and Quantum Sensors programs, among others.

Business innovation

• NRC IRAP International funded an average of 19 new projects a year in Germany during the strategy period (FY 2019-20 to 2023-24), up from 11 during the pre-strategy period (FY 2014‑15 to FY 2018-19).
• These included 12 International Co-Innovation Action Program (ICAP) projects in the strategy's first 2 years, focused on relationship development between Canadian small and medium-sized enterprises (SMEs) and German multinational enterprises (MNEs).

Case study

The AIIR-Power project (FY 2021‑22 to 2023-24) aimed to develop artificial intelligence (AI) techniques for high-voltage photonic power converters used in power and telecom sectors. Formed through an NRC's 3+2 project, it included 4 Canadian and 2 German participants.

According to NRC staff and partners, the project improved NRC–German researcher relationships, advanced solar cell design, validated software models and enhanced AI knowledge for complex optoelectronics, with potential benefits for the energy and telecom sectors.

Japan

International relations

• The NRC's relationship with Japan was assessed as evolving, with deep linkages that are growing broader but are primarily with select partners.
• Executive-level interactions, including minister-to-minister and president-to-president, were essential. NRC staff noted that the extensive experience of the Tokyo office staff facilitated engagement with Japanese organizations.
• Active MOUs increased from 1 to 4 over the strategy period.

Research excellence

• The NRC signed an average of 5 new CRAs per year with Japan, up from a baseline of 1 in FY 2018-19.
• Collaborations included Challenge programs such as Aging in Place, Disruptive Technology for Cell and Gene Therapy and Quantum Sensors.

Business innovation

Before the international strategy, NRC IRAP International did not fund any projects with Japan. During the strategy period (FY 2019-20 to 2023-24), NRC IRAP International averaged 14 new projects per year in Japan. These included 6 ICAP projects in the first 2 years, focused on developing relationships between Canadian SMEs and Japanese MNEs.

Case study

Trilateral projects between the NRC, Concordia University and Kyoto University's Center for iPS Cell Research and Application (CiRA) (FY 2022-23 to 2025-26), under the NRC's Cell and Gene Therapy Challenge program, aimed to accelerate induced pluripotent stem cell therapy for cancer and other diseases using engineered cells from a patient's own body.

According to NRC staff and partners, these projects fostered new Canada-Japan researcher relationships, advanced AI for gene editing, increased the NRC's visibility at CiRA, initiated new CiRA collaborations and trained Canadian students.

United Kingdom

International relations

• The NRC's relationship with the UK was assessed as evolving to mature, with broad research linkages with academic institutions, but business partnerships were limited to specific sectors.
• NRC staff and partners noted that the UK and Canada share similar research and innovation approaches, with aligned research priorities.
• Active MOUs increased from 1 to 2 over the strategy period (FY 2018-19 to 2023-24).

Research excellence

• The proportion of all agreements that were CRAs increased from 5% in FY 2018-19 to 23% in FY 2023-24.
• The NRC collaborated with the UK on its Artificial Intelligence for Design and Quantum Sensors Challenge programs.

Business innovation

NRC IRAP International averaged 14 new projects per year with the UK during the strategy period (FY 2019-20 to 2023-24), up from an average of 5 in the pre-strategy period (FY 2014-15 to 2018-19). These included ICAP projects in the first 2 years, focused on developing relationships between Canadian SMEs and British MNEs.

Case study

The Canada-Inuit Nunangat-United Kingdom Arctic Research Programme (CINUK) (2021 to 2025) included 13 projects with partners such as Inuit Tapiriit Kanatami, United Kingdom Research and Innovation, Polar Knowledge Canada, Parks Canada and the Fonds de recherche du Québec. The program aimed to study climate impacts on Inuit Nunangat's environments and health.

According to NRC staff and partners, the projects produced operational ice maps, carbon emission reduction strategies, satellite technology, engagement with Inuit communities and a shared understanding of Inuit Nunangat's challenges and opportunities.

Appendix B: Methodology

Bibliometric study

A bibliometric assessment was conducted to compare co-publications, subject areas and the bibliometric impact of these co-publications during the strategy period (2019 to 2023) with those from prior to the strategy's implementation (2014 to 2018).

Data review

Corporate data was sourced from nBoss and SAP systems. Grants and contributions (G&Cs) and operations and maintenance funding data were obtained from tracking tools. The data review also included NRC IRAP International and Collaborative Science, Technology and Innovation Program (CSTIP) administrative data, as well as financial and human resources data.

Key informant interviews

Interviews were conducted with NRC staff and external partners to collect information on the value, progress and implementation of the strategy. A total of 49 interviews were conducted:

• 28 with NRC staff
• 21 with external collaborators and stakeholders

Document review

Internal documents and grey literature were reviewed to understand the scope and evolution of the strategy. Documents included:

• strategy documents
• a performance measurement framework
• presentations
• terms of reference
• meeting minutes
• reports

Case studies

Case studies focused on understanding the role of the strategy in, and impacts from, 3 international projects (1 from each of the ecosystem economies):

• AIIR‑Power
• Japan trilateral projects
• CINUK

Appendix C: Limitations and mitigation

Minor methodological limitations experienced in this evaluation, and mitigation strategies, included the following:

Subjective international reputation goal

Progress on the strategy's goal of enhancing the NRC international reputation was difficult to assess, given that reputation is subjective and hard to quantify.

Mitigation: The assessment of this goal was based on key performance indicators identified in the performance measurement framework, which focused on engagement, missions, networks and international MOUs. This assessment was supplemented with additional data sources, including reports, key informant interviews and case studies.

Limited impacts from case studies

Most of the projects supported by the NRC's international strategy had only recently been completed at the time of the evaluation. This posed a challenge in assessing the full range of outcomes of these activities, including longer-term impacts.

Mitigation: Findings from the case studies were combined with other lines of evidence to assess the initial impacts of the strategy.