Evaluation of the Nanotechnology Research Centre

Information
Status: Active
Effective date: July 5, 2022
Prepared by: Office of Audit and Evaluation, National Research Council Canada
Approval: NRC's President
Cat. No.: NR16-403/1-2022E-PDF
ISBN: 978-0-660-45451-1
Related document: Summary Report - Evaluation of the Nanotechnology Research Centre
Alternate format: Evaluation of the Nanotechnology Research Centre (PDF, 1 MB)

Abbreviations and acronyms

BMN
Biomedical nanotechnologies
DAM
Developmental and analytical microscopy
DAT
Detection and automation
DRDC
Department Research and Development Canada
EBP
Employee benefits plan
EDI
Equality, Diversity and Inclusion
EME
Energy, Mining and Environment (NRC Research Centre)
FTE
Full-time equivalent
FWCI
Field-weighted citation impact
GBA Plus
Gender-based Analysis Plus
HHT
Human Health Therapeutics (NRC Research Centre)
IP
Intellectual property
NANO
Nanotechnology Research Centre (NRC Research Centre)
NDC
Nanomaterial depositions and characterizations
NI
Nanotechnology Initiative
NINT
National Institute for Nanotechnology (precursor to NANO)
NRC
National Research Council Canada
OAE
Office of Audit and Evaluation
OGD
Other government department (Canadian federal)
PDFs
Post-doctoral fellows
PRC
Peer review committee (convened for evaluation)
RC
Research Centre (NRC unit)
RCAB
Research Centre Advisory Board
R&D
Research and development
RO
Research Officer
SDT
Security and Disruptive Technologies (NRC Research Centre)
SME
Small- to medium-sized enterprise
STEM
Science, Technology, Engineering, and Math
T&M
Theory and modelling
TRL
Technology readiness level
U of A
University of Alberta
WIN
Waterloo Institute for Nanotechnology

Introduction

An evaluation of the Nanotechnology Research Centre (NANO) was launched in 2021. It assessed the research centre’s transition from the National Institute for Nanotechnology (NINT) to a NRC research centre, as well as the scientific excellence, impact, relevance, stakeholder engagement, and capabilities of the research centre. This report provides main findings and conclusions of the evaluation as well as recommendations.

This evaluation of NANO covered fiscal years 2017-2018 to 2021-22, all years since its inception as a NRC research centre.

NANO transitioned from NINT, which was a joint initiative from 2001 to 2016 of the University of Alberta (U of A), the Government of Alberta, and the Government of Canada; with the NRC as the operator. An evaluation of NINT was conducted in 2016. It is noted that the COVID pandemic was ongoing throughout 2020-21, NANO’s 4th year of operation, and continued in 2021-22.

Evaluation approach

Approach

The evaluation applied a mixed-methods approach, incorporating qualitative and quantitative lines of evidence. A gender-based and diversity lens was also applied throughout the evaluation and was considered in the development of the methods and in the selection of peer review committee (PRC) members.

Scope

The evaluation covered all activities of NANO since its inception, including internal projects, strategic research collaborative projects, and technical services with clients and collaborators. It also examined the implementation of the Nanotechnology Initiative (NI), a project-based formal bilateral collaboration with the University of Alberta (U of A).

Methods

The evaluation included the following lines of evidence:

  • document review (internal and external sources)
  • data review (administrative, financial, project, and performance data)
  • bibliometric analysis
  • interviews (n=51)
    • Staff (n=16)
    • Clients (n=22) and stakeholders (non-clients) (n=13)
  • staff online survey (n=48; response rate of 60%)
  • client hybrid survey (n=52 of 64; 83% of NANO client base)
    • online survey (n= 19, or 36%)
    • client interview questions (n=33; or 62%)
  • project case studies (n=7)
  • peer review

See Appendix A for detailed information on the methods including limitations and mitigation strategies. See Appendix C for information on the PRC.

Evaluation questions

  1. Transition: To what extent has NANO been successful in transitioning from the NINT collaborative model to a NRC research centre?
  2. Scientific excellence: Building upon and transitioning from NINT’s historical work, to what extent is NANO (positioned to be) a leader in scientific excellence in the areas of developmental and analytical microscopy, biomedical nanotechnologies, and detection and automation?
    1. Is NANO advancing (or positioned to advance) scientific knowledge and discoveries?
  3. Impact: Building upon and transitioning from NINT’s historical work, to what extent is NANO achieving (or positioned to achieve) its longer-term expected outcomes of: business innovation, support to government policy solutions; and, social and environmental impacts?
  4. Research focus: To what extent is NANO focussed in the right areas to ensure its relevance within the Canadian nanotechnology-related research and innovation ecosystem?
  5. Engagement: Has NANO engaged with the most appropriate stakeholders in the most strategic and effective ways?
  6. Capacity: To what extent does NANO have the capacity, competencies, and facilities to achieve its objectives and sustain its operations going forward?
 

Profile

The Nanotechnology Research Centre is located in Edmonton, Alberta, on the campus of the University of Alberta. Its mission is to cultivate nanoscience platforms that lead to technologies and applications to enhance human health, the environment, and society’s technological future. With a full-time staff of 56, plus a complement of students and contracted research associates, NANO draws upon foundational expertise across multiple science disciplines in pursuit of innovation.

Research centre overview

Source(s): Document review

Strategic orientation

NANO aims to lead NRC-wide nanotechnology and nanoscience activities. The research centre leverages partnerships to advance Canadian innovations by deploying cutting-edge nanoscience and developmental microscopy capabilities, through NRC-wide and international collaborations. By doing so, NANO aims to set a national standard in microscopy science capabilities that well represents Canada’s offering internationally.

Longer-term, NANO’s goals involve scientific innovations supporting other NRC research centres and divisions with new technologies and techniques cascading into Canadian industry sectors.

Expertise and research focus

NANO focusses on nanoscience and enables technology innovations by conducting collaborative research projects, and offering technical services and access to its facilities.

NANO has 3 main nanoscience research platforms with supporting nanomaterial depositions and characterizations (NDC), and theory and modelling (T&M) capabilities:

  • Biomedical nanotechnologies (BMN)
  • Detection and automation (DAT)
  • Developmental and analytical microscopy (DAM)

Nanotechnology Initiative (NI)

NANO continues, through the NI, the NRC’s long‑standing research partnership with the U of A. The NI is a formal bilateral collaboration agreement between the NRC and the U of A, created following the dissolution of NINT.

Designed to expand Canadian nanotechnology expertise, capacity and capability, and foster breakthrough research, the NI includes joint investments for projects aligned with NRC strategic priorities. 3-year collaboration projects are selected by a committee jointly chaired by NANO and the U of A. Round 1 projects began in 2018, Round 2 projects are scheduled for 2022, and Round 3 projects for 2025.

NANO’s 5-Year Strategic Goals (2019-2024)
  1. Build an internationally recognized, diverse, inclusive, and enthusiastic workforce
  2. Achieve long-term sustainability of NANO's leadership and capabilities
  3. Foster thriving, collaborative relationship with the U of A in NRC programs
  4. Secure national leadership in developmental and analytical microscopy R&D
  5. Hold high-quality intellectual assets in a few, focused nanoscience platforms

Research centre structure

Source(s): Document review

NANO is under the NRC’s Emerging Technologies division. It obtains strategic guidance from its Research Centre Advisory Board (RCAB), which comprises members from government, academia, and industry. The members of the RCAB provide independent and neutral expert advice to the research centre’s Director General.

5 research teams are led by the Director of R&D and supported by the Director of Operations. 3 teams provide a strategically designated area of research focus, while 2 provide supporting capabilities across all teams.

Figure 1. NANO Organizational Structure (2021-2022)
NANO Organizational Structure
Figure 1 – Text version

This illustrates the Nanotechnology Research Center Structure in a 4-tiered hierarchy.

At the top tier is the Vice-President of the Emerging Technologies Division.

The second tier is the Director General: Nanotechnology Research Center and the Nanotechnology Research Center Advisory Board (RCAB), which advises the Director General, NANO.

The last tier includes 5 teams:

3 teams with a designated area of research focus;

  • Developmental and Analytical Microscopy (DAM), with 1 team lead, 6 research officers, and 4 technical officers
  • Biomedical Nanotechnologies (BMN), with 1 team lead, 4 research officers, and 8 technical officers
  • Detection and Automation (DAT), with 1 team lead, 6 research officers, and 4 technical officers

2 teams with supporting capabilities;

  • Nanomaterial Deposition and Characterization (NDC), with 1 team lead, 4 research officers, and 5 technical officers
  • Theory and Modelling (T&M), with 3 research officers
 

Human resources

Source(s): Data review

As of March 31, 2022, NANO had 56 full-time equivalents (FTEs) and 23 students and research associates (RAs) who work on a term basis (students during their education, RAs within 5 years of graduation). This represents a decrease in 2 FTEs (compared to 2017-18) that was offset by the addition of 16 term-based staff.

NANO’s 24 research officers include 4 team leads and other principal investigators. 8 research officers hold adjunct professorships (7 at the U of A and 1 at the University of British Columbia). These adjunct positions strengthen engagement with academic collaborators and provide opportunities to access additional students and third-party funding.

NANO added capacity with research associates and students, 2017-18 to 2021-22

  • 2 Management (-1)
  • 5 Administration (-1)
  • 25 Research Officers (-2)
  • 24 Technical Officers (+1)
  • 13 Students (+2)
  • 10 Research Associates (+10)

NANO met its target for representation of visible minorities, keeping pace with the increasing participation of visible minorities in its field.

NANO remains below target however for representation of women, particularly in leadership roles (i.e. management, team leads).

Visible minorities are adequately represented, but women and other groups are under-represented

Visible minorities are adequately represented, but women and other groups are under-represented
Employment Equity Group 2017-18 2020-21
TargetFootnote* Actual TargetFootnote* Actual
Visible minorities 23% 23% 28% 31% (+8%)
Women 41% 28% 42% 29% (+1%)
Indigenous people 2% Footnote** 2% Footnote**
Persons with disabilities 4% Footnote** 9% Footnote**
 

Financial resources

Source(s): Data review

Salary growth is increasing total expenditures

NANO’s annual expenditures averaged $10.6M, for a total of $53.1M over the evaluation period (2017-18 to 2021-22).

  • Salaries (including the employee benefits plan (EBP)) averaged $7.7M per year, representing the majority of expenditures.
  • Minor capital expenditures averaged $0.5M per year (minor capital investments are those under $350k).
  • Other operating costs averaged $2.4M per year and included equipment maintenance, supplies, and professional services.

There were no major capital investments in NANO facilities since its inception (major capital investments are those over $350k).

Figure 2. Salaries are driving increasing costs
Figure 2 – Text version

A horizontal line chart representing the Nanotechnology Research Center’s salary costs between the years 2017-2018 and 2021-2022.

  • Over the 5 year span, salaries including EBP increased from $6.4M to $8.8M.
  • Minor capital increased from $0.4M to 0.7M.
  • Other operating costs decreased from $2.8M to $2.0M.
 

NANO yet to meet its $1M annual revenue target

Annual revenue averaged $0.75M per year over the evaluation period, falling short each year of its $1M target. It earned more than half of its total $3.8M from strategic R&D projects; while the remainder came from technical service projects, and grants and contributions from Alberta Innovates, a provincial research and innovation agency that previously supported NINT.

Figure 3. Most revenue earned from strategic R&D projects
Figure 3 – Text version

A horizontal line chart representing the Nanotechnology Research Center’s revenue earned between the years 2017-2018 and 2021-2022.

  • Over the 5 year span, revenue from strategic R&D projects fluctuated, increasing from $300K to $510K in 2018-19 and decreasing to $407K in 2021-22.
  • Revenue from technical services decreased from $366K to $169K in 2018-19 and increased to $253K in 2021-22.
  • Grants & Contributions revenue increased to from $100K to $207K in 2019-22 and decreased to $0K in 2021-22.
 

What are strategic R&D and technical services?

Strategic R&D consists of collaborative research projects undertaken with partners to de-risk R&D and accelerate commercial development timelines. Technical services consist of projects that assist clients in solving immediate technical problems (e.g. testing and certifications, prototyping, demonstrations, scale-up).

Facilities

Source(s): Document review, Facilities Review

NANO’s facilities are located on the campus of the U of A in Edmonton, Alberta, spanning 21,330 m². The NRC leases the building from the U of A. These NANO facilities are used for collaborative research projects and offer self-serve or fee-for-service access to equipment.

Additionally, NANO also hosts the Innovation Centre, a rental space on its fourth floor that is open to industry. Featuring 15 rental units, this space provides close proximity to the high concentration of nanotechnology expertise at NANO and the U of A. Also, the U of A sub-leases the fifth and sixth floors back from the NRC for research space and engineering-base work.

NANO’s facilities are complemented by U of A facilities, access to which is governed by NANO and the U of A’s NI and other agreements. Access to the U of A nanoFAB (Fabrication and Characterization Centre) is important for all NANO facilities and teams.

  Developmental and Analytical Microscopy (DAM) Biomedical Nanotechnologies (BMN) Detection and Automation (DAT) Nanomaterial Depositions and Characterization (NDC)
NRC-NANO facilities
  • Environmental Transmission Electron Microscope (E-TEM)
  • Focussed Ion Beam (FIB)
  • Probe/Force Microscopies
  • 9 EMs total
  • Bio Level 2 Microfluidics Lab
  • Chromatographies
  • Cryo-Electron Microscope
  • Organic Chemistry Lab
  • Nuclear Magnetic Resonance (NMR) Spectroscopy
  • Electronics Lab
  • Photonics Lab
  • Bio Level 2 Microfluidics Lab
  • Analytical Chemistry Lab
  • Atomic Layer Deposition
  • Clean Room
  • Mechanical Testing
  • X-Ray Diffraction / Small-Angle X-Ray Scattering (XRD/SAXS)
  • Routine Electron Microscopy
U of A facilities
  • Scanning Probe Microscopy (SPM) Lab (Physics)
  • nanoFAB - Fabrication and Characterization Centre
  • Rf measurement facility (Electrical engineering)
  • Translational Medicine Lab (Medicine)
  • Prion Lab (Physics / Medicine)
  • nanoFAB - Fabrication and Characterization Centre

Clients and collaborators

Source(s): Data and document review

NANO worked with or signed agreements with 63 organizations between 2017-18 and 2020-21. This included 53 unique clients and collaborators that NANO worked with to implement 125 projects, with the majority (63%) located in Alberta. NANO also signed agreements with an additional 10 universities to structure and coordinate future collaborations. Notably its agreement with the Waterloo Institute for Nanotechnology (WIN) at the University of Waterloo led to a series of projects to be launched in 2022-23.

Other government departments

  • 3 clients and collaborators
  • $1.4M in revenues

7 projects with Department of National Defence, Agriculture and Agri-Food Canada (AAFC) and Natural Resources Canada (NRCan).

  • Department of National Defence - Defence Research and Development Canada (DRDC) provided $1.4M in revenues while AAFC contributed in-kind support
  • Collaborative project with NRCan via the NRC’s Energy, Mining and Environment (EME) research centre

Academia and others

  • 11 clients and collaborators plus an additional 10 universities that signed agreements with NANO to collaborate
  • $0.37M in revenues

18 projects with 2 academic clients and collaborators.

  • Most academic projects were with the U of A, including 9 NI projects.

8 additional projects with 9 other organizations and individual researchers, including Albertan provincial agencies and 1 American federal agency.

Industry

  • 39 clients and collaborators
  • $0.71M in revenues

92 projects with industry clients and collaborators.

  • Most industry projects ($0.67M in revenues) were with 37 Canadian industry clients.
  • 86% of Canadian industry clients were small-to-medium enterprises (SMEs).
  • Small share of projects ($0.05M in revenues) with 2 foreign businesses.

Transition from NINT to NANO

Despite significant leadership and change management challenges at the outset, NANO had some success transitioning from the NINT model to a NRC research centre (RC), but the transition is ongoing.

Finding

NANO’s organizational structure, governance, and processes are now consistent with other research centres. Though some stakeholders preferred the old model, NANO now has greater control over its direction.

Source(s): Data and document review

Limitations of the previous model

NINT was a nanotechnology research and development facility based in Edmonton. It operated as a joint initiative between the U of A, the Government of Alberta, and the Government of Canada, through the NRC.

According to the NINT Evaluation (2016), NINT’s alignment to NRC priorities had been largely in theory and not in practice. R&D was strongly influenced by U of A cross-appointees, leading to academic-style research. NINT had limited industrial engagement and was limited to companies in Alberta. Given this, the NRC sought a new model which led to NINT being dissolved and replaced with NANO, a NRC research centre.

Governance and activities better aligned with the NRC

The organizational structure at NANO is now consistent with that of other NRC RCs. The RC reports directly to the Vice President, Emerging Technologies, the RCAB has been established, and NANO uses the current NRC accounting, financial/human resources and performance management systems.

NANO now has greater control over its project selection. It can now prioritize engagement with other clients and collaborators, and participation in NRC programs. NANO’s relationship to the U of A was maintained through the NI that was developed with the aim of having projects that support NRC-NANO priorities.

Mixed views among legacy clients and stakeholders

Equal shares of those who worked with NINT reported an improved experience with NANO (30%) versus a worse one (30%). Per follow-up questions, respondents were split on whether the new model introduced more bureaucracy and hurdles to engagement.

Figure 4. Governance models
Figure 4 – Text version

This illustrates governance model changes that occurred due to the transition of the National Institute for Nanotechnology (NINT) into the Nanotechnology Research Center. The NINT model existed from 2000-01 to 2016-17 and shows that there was large collaboration between the University of Alberta, the Alberta Government and the NRC.

The NANO model was introduced in 2017-18 and shows that the NANO Research Center is run by the National Research council, with a Nanotechnology Initiative collaboration program with the University of Alberta.

 

Staff perspective on transition

Finding

Most NANO staff, including those from the NINT-era, consider the transition as positive. Staff feel connected to the NRC as a whole, take greater pride in their work and have more opportunities to collaborate beyond the U of A.

Source(s): Document review, internal survey, and external and internal interviews

Transition benefits outweigh challenges

Staff surveyed and interviewed had generally positive views of the transition. Among those NINT-era staff (who worked under both models), over half (57%) agreed the transition had been successful versus a quarter (26%) who thought the process was still ongoing.

Even still, an overwhelming majority of NANO staff (86%) felt a sense of belonging to the NRC. This trend that was consistent among NINT-era staff as well as more recent hires. By all indicators, staff were much likelier to report improvements than challenges.

Figure 5. Staff felt the transition to NANO RC had positive impacts

Source(s): Internal survey, NINT-era staff only (n=30).

Figure 5 – Text version

A positive negative horizontal bar chart depicting survey results from 30 National Institute of Nanotechnology (NINT)-era staff regarding the transition from NINT to the Nanotechnology Research Center (NANO).

The results indicate that the transition generally had positive impacts. The surveyed indicators included:

  • Pride in Research and Development projects, with 53% stating an improvement and 3% stating a lower amount of pride.
  • Excellence of projects/research, with 47% stating an improvement and 13% stating a worsened effect.
  • Opportunities for advancement, with 43% stating an increase, and 10% stating a decrease.
  • Opportunities for training, with 40% stating an increase and 7% stating a decreas.
  • Expertise aligns with objectives, with 40% stating an increase and 10% stating a decrease.
  • Work-life balance, with 23% stating an increase while 10% reported a decrease.
 

More opportunities to collaboration

During the NINT-era, contributions to the work of other RCs and programs were limited. NANO staff surveyed for this evaluation felt that they now had more opportunities to collaborations beyond their RC. Nearly two-thirds of NINT-era staff felt they have more opportunities to engage other research centres, with about one third of all NANO staff reporting that they ‘often’ or ‘always’ engage other RCs (e.g. collaborative projects, expertise and facility sharing).

Collaborators from other RCs indicated that they are often engaged in multiple projects with NANO and noted well-established relationships and reliance on NANO’s capabilities.

Figure 6. More opportunities to collaborate within the NRC and outside Alberta

Source(s): Internal survey, NINT-era staff only (n=30)

"More opportunities to collaborate within the NRC and outside Alberta "
Figure 6 – Text version

A positive negative bar chart depicting survey results from 30 National Institute of Nanotechnology (NINT)-era staff regarding opportunity for collaboration as a result of the transition from NINT to the Nanotechnology Research Center (NANO). The results indicate that the transition increased the amount of opportunities for collaboration. The surveyed indicators included:

  • Within the NRC (other Research Centers), with 63% finding better opportunities and 3% finding fewer opportunities.
  • Nationally (outside Alberta), with 53% finding better opportunities and 0% finding fewer opportunities.
  • Internationally, with 27% finding more opportunities and 7% finding fewer opportunities.
  • Alberta (outside of the University of Alberta), with 27% finding more opportunities and 10% finding fewer opportunities.
  • University of Alberta, with 23% finding more opportunities and 30% finding fewer opportunities.
 

Resource changes

Finding

The transition empowered NANO with greater control over its research strategy and operations, but with diminished ability to leverage resources from the U of A.

Source(s): Document and data review, and internal interviews

Transition led to leadership gaps and loss of students

The loss of cross-appointees from the U of A, post-transition, left a gap in scientific leadership and led to a reduction in students. Some interviewees felt losses were not well-considered in planning, resulting in leadership gaps. Some also attributed the decrease in publications to the loss of cross-appointees, and their associated students and stature. Once its new structure and strategy were in place in 2019-2020, NANO mitigated these effects by promoting from within to fill lead roles.

The PRC noted that, as NANO transitioned from NINT, it was difficult to form cohesive teams because it had researchers from differing backgrounds with specific expertise. Nevertheless, NANO did form teams under research themes to have an impact on new or improved nanotechnologies, applications and solutions. The research centre continues to have challenges hiring and retaining highly-qualified personnel.

Facilities transition includes access agreements

The NRC and the U of A agreed on the ownership and location of equipment bought by NINT. For example, the SPM Laboratory was located at the U of A and remained at the university. NANO accesses this laboratory via the NI agreement. The NRC has custodianship of BMN, DAM, DAT and the NDC facilities.

Resource transition from NINT to NANO
  NINT
(2001-2016 average)
NANO
(first year 2017-18)
Human resources
  • 78 FTEs
  • 50 post-doctoral fellows (PDFs)
  • 120 students
  • 59 FTEs
  • 0 PDFs
  • 10 students
Budget
  • $12.9M
  • $9.96M
Competencies/teams
  • Materials, Fabrication and Characterization of Nanostructures
  • Surface and Interface Science
  • Electron Microscopy
  • Enabling Facilities and Policies
  • Biomedical Nanotechnologies
  • Detection and Automation
  • Developmental and Analytical Microscopy
  • Nanomaterial Depositions and Characterizations
  • Theory and Modelling

"Even though resources left, the NRC gained better control, better leadership."

NANO staff

Implementation of the Nanotechnology Initiative (NI)

Finding

The Nanotechnology Initiative (NI) has been implemented but has experienced some challenges. There remains a need to better communicate project selection and decision-making processes.

Source(s): Document review, and external and internal interviews

Round 1 projects were misaligned and delayed

NI round 1 projects, launched in 2018-19, were not fully aligned with the research centre’s strategic objectives, nor well aligned with the strategic mandate of the NRC. In the NANO project selection process, projects are scored against weighted criteria.  The weight assigned for alignment with NRC’s strategic mandate was very low (5%) as NANO did not have a strategic plan at the time.

Stakeholders generally lacked confidence in the project selection process. Although project selection criteria and weighting were clearly defined, there was limited documentation outlining the selection process for NI round 1 (e.g. lack of clear rationale for acceptance and rejection of proposals).

The implementation of round 1 projects was delayed due, in part, to initial issues hiring post-doctoral fellows (PDFs).

Round 1 project topics:

  • adaptive materials
  • diamond nanophotonics
  • energy storage
  • Graphene in All-New Nanodevice Technologies (GIANNT)
  • Immunoglobin E (IgE)-based immunotherapy
  • molecular junctions
  • nanofluidics
  • nano-optomechanical
  • photovoltaics computation

Stakeholders are optimistic round 2 projects will be better aligned

With NANO’s strategic plan now in place, weighting for alignment with NRC’s strategic mandate has been increased to 30% which guided the selection of projects for Round 2. In addition, a notice of intent phase was added to the process (preceding proposal submissions).

Some NANO interviewees (interviewed prior to Round 2 project selection) were optimistic that Round 2, which launched in 2021-22, would better align with NRC priorities and be less problematic than Round 1. However, given that project selection documentation was unavailable, the analysis of Round 2 project alignment was not possible.

NI value added, compared to other collaborations, is uncertain

Some internal staff reported that their collaboration projects with the U of A had proceeded through other means even though they were not selected by the NI. Thus, the added value of the NI framework, compared to other collaborations between the 2 organizations is not clear. This was questioned by internal and external stakeholders, now that other collaborative opportunities exist (e.g. NRC Challenge Programs, Industry Science and Economic Development’s Supercluster Programs).

Scientific excellence

NANO is positioned, to some extent, to be a leader of scientific excellence and advance scientific knowledge and discoveries. It is most recognized as a national leader in developmental and analytical microscopy, and less known for its other research platforms and supporting capabilities.

Scientific leadership and recognition

Finding

NANO is considered by over half of its clients and stakeholders as a leader. However, this is not consistent across all research areas, with NANO’s work in microscopy standing out as a leader.

Source(s): Client survey, data review, external and internal interviews, and peer review committee

Clients consider NANO a leader, mainly in microscopy

Most of NANO’s clients and stakeholders consider it to be a national leader in at least 1 research area. Recognition by research area varies, with the strongest level of leadership being in DAM. In that area, NANO has led the creation of the NRC Microscopy Network, and NANO staff have fulfilled executive roles with the Microscopical Society of Canada. NANO leadership also participate in the boards of key organizations in the ecosystem (e.g. McMaster University’s Canadian Centre for Electron Microscopy). Across areas, NANO staff have demonstrated leadership by hosting workshops, lectures and virtual events.

Limited international recognition

Less than a third of clients and stakeholders would consider NANO an international leader. To their credit, NANO staff have presented to industry groups and international conferences (including in Germany, Japan, Korea and the United States) and participated on scientific committees such as ISO standards committees. However, NANO has not routinely tracked conference or committee participation which limits the ability to provide a complete picture of these efforts.

Peer review found NANO positioned to lead across areas

Per the peer review, NANO caters to a wide audience comprising both academic and industrial stakeholders and offers a very good breadth of expertise in closely related NANO disciplines. Supporting this are teams with the training and capacity to achieve a good level of excellence, and led by scientists with good reputation.

Figure 7. Recognized as a scientific leader by over half of clients and stakeholders

Source(s): Client survey (n=48)

The question is structured so international leadership implies national as well.

"Recognized as a scientific leader by over half of clients and stakeholders"
Figure 7 – Text version

An illustration of survey results from 48 clients regarding NANO’s status as a scientific leader. The survey results indicates that:

  • 60% of clients and stakeholders consider NANO and a scientific leader in at least 1 area.
  • 29% view NANO as a leader internationally in at least 1 area.
  • 31% view NANO as a leader nationally in at least 1 area.
  • 40% view NANO as not leading in any areas/unsure.
 

Contributions to scientific and technological advancements

Finding

NANO has made contributions to scientific and technological advancements, although performance indicators to measure this could be improved.

Source(s): Client survey, data review, and external and internal interviews

Contributions made to scientific and technological advancements

NANO contributed to scientific and technological advancements, although, as expected, outcomes vary between research collaborators versus technical service users. Collaborators were more likely to report new discoveries or contributions to advancing science through publications, while service users were more likely to report increased R&D capacity or advances in technology development. Given the delays in NI round 1 projects, it is too soon to comment on their impacts, beyond outputs such as presentations and publications.

Figure 8. Contributions to clients and collaborators

Source(s): Client survey (n=35)

"Contributions to clients and collaborators"
Figure 8 – Text version

A horizontal bar chart depicting survey results concerning NANO’s contributions to clients and collaborators. 35 clients and collaborators were surveyed using 4 indicators:

  • new discoveries or achievements (51% reported contributions)
  • increased Research and Development capacity (40% reported contributions)
  • advanced technology development (37% reported contributions)
  • contributed to publications (31% reported contributions)
 

Non-academic performance indicators recommended by the PRC

The PRC recommended that NANO measure and monitor more industry and impact-oriented performance indicators, beyond academic ones (e.g. publications, citations). Project case studies identified instances of such academic and non-academic contributions to clients and collaborators and examples are shown below.

Nano-enabled DNA vaccine delivery

Biomedical team with DRDC:

  • tested and advanced new approach to targeted cell delivery with potentially broad applications
  • established inter-disciplinary network to facilitate future collaborations
  • co-authored publication (forthcoming)
Microfluidic device prototyping

DAT team with a Canadian SME:

  • the SME lacked in-house expertise in microfluidics for engine technology
  • NANO designed and delivered prototype for sorting fine particles suspended in oil to reduce wear
  • next phase will enhance performance
Nanostructured energy storage

Nanomaterials team with the U of A (via NI):

  • addressed disconnect between academic and industrial approaches
  • led to 6 peer-reviewed publications, and eleven speaking invitations, hosted visits and workshops with academia and industry

Intellectual property generation

Finding

NANO has continued to generate new intellectual property (IP) and is looking to exploit new opportunities to meet targets.

Source(s): Client survey, data review, and external and internal interviews

NANO met IP targets most years, plans to grow

Since 2017-18, NANO has generally met its annual IP targets with a focus on patent applications in Canada, Japan and the United States.

While the invention disclosure rate has remained steady, patent activity has waned in the past 2 years. NANO plans to correct this and in fact increase IP generation by 25% by 2023-24. They plan to achieve this in part from opportunities created by its new Microscopy Network.

Low citation rate relative to the NRC, Canada overall

NANO’s patent citation rate (11.6) lags the NRC (27.6) and the Canadian (17). This may be attributed in part to the loss of capacity and prominence when NINT was dissolved and high profile cross-appointees departed.

Figure 9. Invention rate steady, patent rate down

“Patents Issued” includes both applications filed by NINT and more recently by NANO.

"Invention rate steady, patent rate down"
Figure 9 – Text version

A clustered column chart comparing invention disclosures, patent applications and patents issued between 2017-2018 and 2020-2021.

  • The amount invention disclosures per year remained steady between 4 and 5 from 2017-18 to 2020-21.
  • The amount of patent applications decreased from 8 to 5 from 2017-18 to 2020-21.
  • Issued patents decreased from 4 to 1 from 2017-18 to 2020-21.

“Patents Issued” includes both applications filed by NINT and more recently by NANO.

 

Examples of patents issued (2017-18 to 2020-21):

  • method of fabricating nanotips with controlled profile (Canada)
  • devices and methods for nanoparticle enhanced impedance-based molecular sensing (US)
  • nanotubes as carriers of nucleic acid into cells (US)
  • synthesis of aliphatic polyester/polycarbonate triblock copolymers via organocatalytic neat ring opening polymerization (Japan)

Scientific knowledge generation

Finding

NANO has not met its publication targets, mainly as a result of its reduction in staff and has lost scientific influence. NANO has signed new agreements with universities which may help to reverse this trend and support more scientific knowledge generation.

Source(s): Data review (including bibliometric analysis), external and internal interviews, and peer review committee

Reduced publishing capacity

Since the transition, NANOs scientific influence and impact has decreased mainly because of the loss of cross-appointees and PDF support from the U of A. NANO did not meet its publication targets in recent years.

NANO’s scientific impact is below Canadian averages for its areas of focus with an overall field-weighted citation impact (FWCI) of 1.12. This is below NRC’s average of 1.35 for the same period (2017 through mid-2021). Looking at examples of its publications however, the PRC did feel NANO had published impactful research. They found the quality and scientific impact of the research conducted at NANO in the areas of focus was evidenced through publications in high impact journals.

New agreements create co-authorship opportunities

Nearly all of NANO’s publications feature external co-authors (95%), a rate higher than averages for the NRC (81%) or Canada overall (70%). However, co-authors remain heavily concentrated at the U of A (84% of publications feature U of A staff). New academic agreements, in addition to the NI, may be a means to collaborate and co-author with other leaders in the field to have a greater impact. NANO may consider setting co-authorship targets with its new university partners to prioritize knowledge generation.

Figure 10. Averaging 37 publications per year below annual targets

Targets not available for 2017

"Averaging 37 publications per year below annual targets"
Figure 10 – Text version

A multiple width overlapping column chart describing target amount of publications and actual amount of publications by the NANO Research Center from 2017-2020. Additionally, there is a table outlining citation scores from the 5 NANO teams compared to Canada as well as the World.

The actual amount of publications is consistently below the target amount of publications for all years recorded. The overall citation score for NANO 2017-2020 is 1.12, which is higher than the world average of 1.00 but lower than the Canadian average of 1.45.

 
Citation scores relatively low
Team FWCI
(2017-2021)
NANO Canada World
DAM 1.16 1.82 1.72
BMN 0.87 1.99 1.83
DAT 1.04 1.48 1.57
NDC 1.25 2.38 1.70
T&M 1.16 0.97 1.13
Overall 1.12 1.45 1.00

The Field-Weighted Citation Impact (FWCI) is a normalized indicator to gauge relative performance of a publication in terms of citations, taking into consideration publication volume and norms in different research domains.

Demonstrated leadership and alignment to mission

Finding

NANO is most well-known and best positioned to lead in Developmental and Analytical Microscopy (DAM), due to its capabilities in this field.

Source(s): Bibliometric study, client survey, external and internal interviews, and peer review committee

Considered a national leader in DAM

Over half of clients and partners (56%) consider NANO a national leader in DAM, though less so internationally (27%). Its DAM facilities and technical expertise were identified by external interviewees as areas where NANO is leading-edge.

Contributing to advancements in microscopy

Through collaborations with academia and industry, as well as internal projects, the DAM team has contributed to advancements in its field. For examples, the team:

  • developed soft material nanostructure imaging via atom force microscopy
  • advanced Cryo EM, leading to a patent filing and catalyzing a local SME to pursue this technology further
  • advanced the development of NanoMI, an open-source transmission electron microscope

Important to fundamental science but under-cited

The PRC considered the NANO’s DAM research to be of high quality and important to fundamental science, with publications having been cited by top organizations in China and France. However, its overall citation score in DAM (1.16) lags Canadian leaders like the University of Toronto (2.48) and the U of A (1.50).

Finding

NANO has improved its position in Detection and Automation (DAT), and most effectively aligned its research activity to the NANO mission.

Source(s): Bibliometric study, client survey, external and internal interviews, and peer review committee

DAT team most aligned with the NANO’s mission

The PRC highlighted the DAT team’s alignment with the NANO mission as a best practice. According to the committee, this team has effectively embraced the NANO mission and is focused on the integration and application of detection platforms with increasing attention to climate sciences and environmental sensing. The PRC agreed that the DAT team is on the right path to easily and quickly grow its influence by engaging with stakeholders in these areas (e.g. Environment and Climate Change Canada (ECCC), NRCan, academia, and industry).

DAT team contributing to, but not leading the field

Few consider NANO a leader nationally (14%) or internationally (4%) in this area. Most clients and partners (73%) were unsure how NANO compares with others in DAT. Nonetheless, a number of advancements were realized. The DAT team:

  • demonstrated that receptor proteins in the body can be used as sensors
  • demonstrated artificial receptors that mimic biological entities for the overdose-reversing drug naloxone
  • supported development of microfluidic sensors that combine molecular self-assembly and electrochemical impedance spectroscopy

Potential to grow and integrate capabilities

Finding

NANO has applied its expertise in Biomedical Nanotechnologies (BMN) to collaborative projects, but has yet to establish itself as a leader in this field.

Source(s): Bibliometric study, client survey, external and internal interviews, and peer review committee

Not leading in BMN

Despite contributions to some advancements in this field, most of NANO’s clients and partners (71%) were unsure how NANO compares with others in BMN. Few considered them to be a national (17%) or international (4%) leader. Moreover, BMN is the one area where NANO is below the FWCI average of 1.0.

Advancing health science with academia, OGDs

The BMN team has worked with government and academia to advance health science. With academia, for examples, the team has worked with the U of A to access a Canadian Institutes of Health Research grant, and more recently launched 4 new projects via the NI. With OGDs, the team worked with DRDC to advance new drugs and vaccine delivery systems to protect against biological threats.

Potential to grow in collaboration with others

Collaborations with other research centres and universities may support growth. The PRC considered the BMN team to have established a reputation of academic excellence and to have engaged extremely well with industry in the past. However, connections with clinical expertise, more specifically with the NRC’s Human Health Therapeutics (HHT) Research Centre, are still needed to move their technologies forward.

Finding

NANO has demonstrated strength in Nanomaterials Depositions and Characterizations (NDC), but needs to better position the NDC as a supporting capability.

Source(s): Bibliometric study, client survey, external and internal interviews, and peer review committee

Demonstrated complementarity

This capability also complements current and expected future NRC academic and industrial collaborations. For example:

  • Cross-platform characterization capabilities enhance other NRC energy-related programs (e.g. EME and SDT) by improving lithium-ion electrode capacity, purity and performance at high temperatures.
  • Cross-platform deposition capabilities advance industrial fabrication methods of new composite and hybrid materials developed in collaboration with the industry and OGDs.
  • Accessible toolset distinguishes NANO’s existing infrastructure, creating unique long-term capabilities for both NANO and the U of A (e.g. x-ray diffraction).

Not fully committed to supporting role

The PRC found the NDC team to be in an awkward position of not being considered one of main platforms yet operating quite independently. For instance, while the team was engaged in sensor development projects in support of the DAT team, it also pursued more opportunistic projects that were not aligned with NANO’s 3 core platforms (either to earn revenue or engage with NRC programs outside of NANO). The PRC recommended NANO act to integrate the NDC team more fully into the NANO strategy so that its contributions to the mission is clear to all.

Impact

NANO has made some contributions to the NRC’s longer-term goals of business innovation and government policy solutions in the environmental and health fields, such as drug screening, vaccine delivery, and hazard detection.

Health, environmental and social impacts

Finding

Its high number of projects (across all NANO teams) in human health, especially on COVID-19, has yielded, and is expected to continue to yield, enhancements in these sectors. Future environmental and societal technological impacts may also be felt, but over a longer timeframe, given the lower technology readiness level (TRL) of NANO research.

Source(s): Document review, external and internal interviews, client survey, and peer review committee

NANO is working to enhance human health, pandemic response

Clients and collaborators, including other government departments (OGDs) and Canadian and foreign organizations, identified projects, across all NANO teams, with potential health applications, including:

  • DAM: advanced imaging of COVID-19 spike proteins
  • BMN: new strategies for DNA delivery and a new drugs and vaccine delivery system
  • DAT: a molecularly imprinted polymers-based electrochemical drug sensor to detect and protect humans from bio-agents and weapons
  • NDC: rapid and real-time detection technology of food-borne pathogens
  • T&M: a new computer calculation based on artificial intelligence/machine learning for enhanced drug screening

Additionally, 3 round 2 NI projects related to health including potential treatments for acquired prion disease, cancer and central nervous system disorders.

NANO contributes to environmental impacts

NANO’s work is contributing, or is well-positioned to contribute, to impacts on the environment through work on renewable energy, battery technology/energy storage, clean energy and pollutant detection. For example, NI photovoltaic research is supporting development of new material devices, using artificial intelligence/machine learning with the potential to improve the performance of hybrid solar cells and to reduce manufacturing costs.

Incremental contributions to societal progress

NANO’s low-TRL work shows limited direct impacts in advancing society’s technological future in the near-term but it is positioned for long-term impacts. For example, NanoMi, NANO’s open-source transmission electron microscopy, may open the door to scientists around the world, enabling access to nanotechnology that would typically require commercial manufacturers, which are difficult to access. In turn, these scientists may advance research that lacks immediate commercial appeal but nonetheless may generate benefits for society.

Government policy solutions

Finding

To date, NANO has made limited contribution to government policy solutions, although it has potential to contribute to Challenge Programs and Supercluster Programs, which could lead to policy solutions including future pandemic and vaccines policy.

Source(s): Document review, external and internal interviews, client survey, and peer review committee

NANO has made indirect contributions to government policy solutions

NANO’s contributions to government policy solutions are often indirect through its support to other NRC research centres who are in direct contact with the OGDs. As well, OGDs tend to view their work with NANO as having the potential to contribute to government policies, regulations or guidelines, rather than report concrete impacts to date. Examples included:

  • Work with DRDC to find innovative ways to protect against and treat biological threat agents and emerging diseases, which could influence national security policy.
  • Work with AAFC on detection of food-borne pathogens, which could influence food safety policy.
  • NI photovoltaic research which could influence clean energy harvesting and development policy.
  • A presentation to the International Standards Organization (ISO) and Standards Council of Canada, which could contribute to development of standards in tomography as well as measurement and characterization.

Positioned to enable greater impact

Exploring opportunities to work directly with more OGDs may raise NANO’s profile and potentially generate revenues. Through its collaboration with other research centres, NANO is positioned to enable government policy solutions (e.g. gene and cell therapies for the Aging in Place challenge program, nano-encapsulation of food additives for the Sustainable Protein Production supercluster).

Business innovation

Finding

NANO is enabling long-term impacts for Canadian businesses by accelerating R&D, mainly in Alberta.

Source(s): Document review, external and internal interviews, client survey, and peer review committee

Contributions to long-term benefits for businesses

Canadian businesses reported that working with NANO advanced R&D and led to new products. However, NANO’s low-TRL contributions can take a long time (longer than the 5-year evaluation period) to demonstrate impact. For examples:

  • A Canadian SME participated in a 3-year project with NINT (ending 2016) to develop a non-aqueous coding process that explored antimicrobial properties of silver compounds. Since, the technology has been commercialized, resulting in job creation and increased sales.
  • More recently NANO worked with a Canadian engineering company that sought NANO’s expertise in microfluidic device prototyping. This firm is now planning a follow-up project with considerations for commercial production.
Figure 11. Canadian businesses report NANO supported innovation

Source(s): Client survey (Canadian industry only, n=15)

"Canadian businesses report NANO supported innovation"
Figure 11 – Text version

A horizontal bar chart depicting survey results concerning NANO’s support for innovation in Canadian businesses. 15 Canadian clients were surveyed using 6 indicators:

  • advanced or accelerated technology development (53% of clients reported support)
  • launched or improved products/services (33% of clients reported support)
  • created new jobs (20% of clients reported support)
  • led to new partnerships (13% of clients reported support)
  • increased funding or investments (13% of clients reported support)
  • increased sales (7% of clients reported support)
 

Business impacts remain heavily concentrated in Alberta

With three-quarters (74%) of its industry clientele in Alberta, NANO has missed opportunities to engage beyond the province to build national R&D capacity. The PRC also recognized the need for NANO to engage more with Canadian industries and explore multi-year, revenue-generating projects.

IP licensing was identified as a barrier

The NRC does not usually grant exclusive use of IP that resulted from public investment. SMEs are typically interested in sharing IP when they invest in collaborative research projects. Industry stakeholders and clients reported that multi-national organizations (which are mainly foreign) are not interested in sharing IP. As a result, industry partnerships can be affected, but is not unique to NANO.

COVID-19: challenges and opportunities

Finding

The COVID-19 pandemic has brought both challenges and opportunities to NANO. Previously planned projects and operations were delayed, but NANO teams contributed to pandemic-related research with other research centres, OGDs and international collaborators.

Source(s): Document review, external and internal interviews

Interruptions and other challenges

All projects reviewed for the evaluation encountered at least some interruptions due to the pandemic, such as:

  • Lab access was interrupted, stopping experimental work which could not be conducted remotely. Attempts to access remote operation technology failed due to NRC security safeguards.
  • Supply chain and other logistical challenges resulted in increased wait times for materials and maintenance. In an extreme case, a $1M spectrometer purchased pre-pandemic was not installed as of fall 2022 as foreign-based technicians could not travel to Edmonton.
  • Travel restrictions also disrupted client communications, making it more difficult to resolve technical issues better addressed in-person.
  • The implementations of all NI round 1 projects were delayed.
  • Hiring of PDFs proved difficult as candidates left for other positions while work at NANO was delayed.

Positive impacts of ‘work from home’ conditions

Pandemic restrictions did have some unintended benefits, including:

  • publications increased by over 20% from 2019 to 2020
  • remote meeting technology supported more engagement with other research centres and virtual attendance at international conferences
  • demand increased for modelling

Opportunities to support Canada’s pandemic response

Several opportunities were seized by NANO to advance virology and related research:

  • The DAM team further developed its electron holography instrument to enable 3-D imaging of spike proteins of COVID-19. In doing so, NANO created a new, more efficient nanoscale microscope with broad applications for human health.
  • The NANO team collaborated on a COVID-19 project with NRCan, RIKEN (a Japanese research organization) and 2 Japanese universities on molecular and theoretical modelling. The project, aimed at developing an alternative treatment for individuals with unique health concerns related to specific ingredients in the COVID vaccines, could have policy implications.
  • NANO has had some notable publications, including one on the impact of SARS CoV-2 on the central nervous system.
  • The T&M team worked with NRC-HHT on drug screening using 3D RISM simulation as part of the NRC’s Pandemic Response Challenge Program.

Research focus

NANO has defined its strategy and mission to respond to national priorities and the needs of stakeholders within the Canadian nanotechnology ecosystem, but work remains to align NANO’s research focus to its stated mission.

Strategic orientation

Finding

NANO has aligned its research to a range of federal priorities and the NRC’s strategic objectives. As well, other research centres rely on NANO’s expertise to make innovative contributions to their respective fields.

Source(s): Document review, external and internal interviews, facilities reviews, and peer review committee

Research aligned with federal and NRC priorities

NANO’s mission is to cultivate nanoscience platforms that lead to technologies and applications to enhance human health, the environment and society’s technological future.

The PRC indicated that the NANO mission seems to be aligned with the Canadian nanotechnology research and innovation ecosystem as it focuses on 3 main research platforms: microscopy, biomedical nanotechnologies, and detection and automation. These platforms aim to complement the NRC’s and Canada’s implementation of nanotechnologies in multiple sectors.

Several projects have been implemented to support federal, NRC and NANO mission priorities in collaboration with other research centres (HHT, SDT, EME, Digital Technologies, and Metrology) and academic partners.

NRC Challenge Programs are federally funded programs designed to bring together researchers and facilities from across the NRC with academic and industrial partners by supporting multi-party R&D that address Canadian priorities.

Federal Superclusters are industry-led, fund matching programs to boost innovation for academia, SMEs, and not-for-profit organisations in a particular sector.

NANO participating in NRC programs aligned with their mission

Human health

Aging in Place Challenge program

  • analysis of exosome production in the aging brain

Sustainable Protein Production Supercluster

  • nanoencapsulation of food additives and flavouring
Environment

Materials for Clean fuels Challenge program

  • functionalized boron nitride nanotubes for energy applications
  • developing machine learning algorithms

Advanced Clean Energy Supercluster

  • next generation materials for solid state batteries
Society’s technological future

Artificial intelligence for the Design Challenge program

  • accelerated MEMS design optimization
  • AI for simulation of biological systems

High-throughput and Secure Networks Challenge program

  • development of low power deformable mirror adaptive optics

Quantum Sensors Challenge program

  • atom‐defined quantum devices

Alignment with the NANO mission

Finding

NANO has taken action to adapt its projects to the new priorities of the NANO mission; however, there are issues regarding the alignment of the teams with these priorities and strategic direction.

Source(s): Document review, external and internal interviews, facilities reviews, and peer review committee

NANO takes action to align with the mission

NANO works with other organisations to identify stakeholder needs, find areas of complementarity, and strengthen its position in the Canadian nanotechnology ecosystem. Currently, NANO is focussing on 3 platforms (DAM, BMN and DAT).

To move its strategic direction forward, NANO aims to allocate its resources evenly among internal capability projects, NI projects, and other external projects.

Project selection processes have changed to ensure project relevance and rectify problems identified by the RCAB. For NI projects, this includes increasing the weight allocated (to 30%) for alignment with the NRC’s strategic mandate.

For other external projects and internal projects, NANO adopted a new project management system, revised project selection criteria, and now considers R&D Director recommendations that prioritize external projects while considering internal capabilities.

"In order to bring added value, NANO’s project portfolio and ultimately everything that NANO does should focus on the NANO mission."

Peer Review Committee

Challenges remain to attain team alignment

The PRC recognized that NANO is willing to adapt projects to the priorities of the NANO mission. However, they also observed issues regarding the alignment of the teams to the NANO mission.

The PRC indicated that the NANO teams have the ability to achieve a level of excellence but may be spread in too many directions. The PRC encouraged NANO to narrow their focus and to not rely on previous partnerships, particularly with the U of A, to connect with new stakeholders.

The PRC underlined the importance of NANO platforms and teams accepting and following the RC’s mission. They indicated that some work seems academic and opportunistic, for example:

  • drug delivery vehicles based on organogels with applications in inflammation may be aligned to other NRC areas
  • some higher-TRL projects in DAM would further the NANO mission
  • NDC has taken on opportunistic projects that are not aligned with NANO platforms

The PRC noted that NANO’s strategic and operational plans require an analysis of current funding needs and opportunities to better align with NANO’s mission.

Achieving a balanced project portfolio

Finding

NANO’s project portfolio has yet to achieve the right balance. This is reflected in project selection and revenue generation. Opportunities for revenue generating projects with SMEs and industry have yet to be realized, and revenues are highly dependent on 1 OGD client, which represents a risk.

Source(s): Data review, document review, and external and internal interviews

Balanced project portfolio, and corresponding increased revenue from industry, required to align with mission

Despite NANO’s actions to adapt to its strategic direction, challenges remain. The PRC indicated that NANO should manage the choice of projects in order to develop cutting-edge nanoscience. For example, the project selection process is insufficiently documented, making it difficult to articulate and justify project choices and understand how projects are aligned with strategic objectives. 

The current model of project selection using 3 equally-distributed categories has not been achieved as too many resources are allocated to lower TRL work (internal and NI).

Figure 12. NANO has yet to achieve balance among internal, NI, and external projects

Note: Operating cost data from 2018-19 to 2020-21. Structure started in 2018-19, therefore information for 2017-18 is not available.

" NANO has yet to achieve balance among internal, NI, and external projects "
Figure 12 – Text version

A Horizontal stacked bar chart illustrating operating costs from 2018-19 to 2020-21. The costs are listed as such:

  • internal (41%)
  • Nanotechnology Initiative (NI) (39%)
  • external (20%)
 

The PRC noted that NANO should increase efforts to go beyond lower TRLs, as a higher TRL focus would be more attractive to industry. Currently, revenue from industry is low ($0.71M from 2017-18 to 2020-21); increasing this amount could both bring in funding and ensure industry relevance. In addition, revenues are highly dependent on 1 OGD ($1.4M from DRDC), which is a risk if the agreement is discontinued.

The PRC suggested that NANO ensure the right balance between lower TRL and higher TRL projects with a view to more engagement with industry and OGDs. This would result in a strategically-selected and balanced project portfolio and ensure an optimal mix of lower and higher TRL projects. To do so, NANO should consider the following dimensions to achieve the right mix:

  • potential TRL impact - low, medium, and high
  • stakeholder level - academia, internal (NANO/NRC capability), and external (industry and OGD)
Figure 13. Nearly half of $3M in revenue from OGDs, 2017-18 to 2020-21
"Nearly half of $3M in revenue from OGDs, 2017-18 to 2020-21"
Figure 13 – Text version

Horizontal stacked bar chart illustrating revenues from 2017-18 to 2020-2021. The revenues are listed as such:

  • other Government Departments (45%)
  • industry (24%)
  • grants and contributions (19%)
  • provincial government (6%)
  • other (6%)
 

Value-added through complementarity

Finding

NANO has identified areas where it may complement other Canadian organizations rather than compete, referencing the uniqueness of some capabilities.

Source(s): Document review, external and internal interviews, facilities reviews, and peer review committee

NANO facilities complement the infrastructure of others

The aim of NANO is to complement the U of A’s Nanofabrication facilities instead of competing with them. A cross-NRC committee has been set up with representatives of all research centres that use related infrastructure. NANO has also set up and leads an Electron Microscopy Community of Practice throughout the NRC which is seen as very favourable by the PRC and other RCs.

The PRC noted that efforts have been made to complement NRC infrastructure rather than duplicate, except when duplication might be necessary based on geographic consideration and purpose of the infrastructure (research vs. technical service). NANO is also making cleanroom space available to companies where they can bring in equipment dedicated to them.

NANO has some unique capabilities

NANO’s clients and partners were split on how unique the research centre is within Canada. Many clients, partners and staff believed NANO is unique in some regard, particularly in terms of its expertise in microscopy. However, other Canadian academic institutions were cited as having comparable or better instrumentation. Half of clients surveyed considered other options, in particular the U of A’s Nanofabrication capability.

However, DAM facilities are highly specialized and relatively unique in Canada in niche areas such as point projection microscope, open-source transmission electron microscope (TEM) and phase plate TEM.

"NANO forming a centralized research facility to partner with academia and industry is hugely important not only because we want to fund local innovation, but because there is such a large economic activation barrier to this type of work. Without NANO, only very large companies can innovate."

External stakeholder

Engagement

NANO is engaging new clients and developing relationships with existing clients and collaborators. However, there is potential for NANO to engage with a broader range of clients (outside of Alberta) and to increase stakeholder awareness of its services/capabilities.

Engaging clients and collaborators

Finding

NANO has established relationships with Canadian stakeholders and top international players; expanded its partnerships with other government departments; and increased collaboration with other NRC research centres. However, there is potential to engage with a broader range of clients outside of Alberta to achieve the right mix.

Source(s): Data review, document review, external and internal interviews, and peer review committee

NANO working to achieve the right mix of stakeholders

The NANO strategic plan and stakeholder engagement plan proposed increasing exchanges with key organizations in Canada and internationally. NANO has diversified its client-collaborator base, engaging more with other sectors including government, academia and other NRC research centres.

NANO has generally met its annual targets for clients, averaging 21 unique clients per year versus a target of 22. In addition, 70% of clients and collaborators were satisfied with their relationship with NANO.

Having said this, 63% of NANO’s client base remains heavily concentrated in Alberta (majority are private sector) and with the exception of electron microscopy, NANO lags the NRC when it comes to international collaboration. The PRC noted that a better client/collaborator/stakeholder mix is needed.

The PRC indicated that the relationship with the U of A should be nurtured, but that it should be kept to approximately one-third of the project portfolio. The PRC acknowledged that NANO promotes engagement with academia through adjunct professorships, and viewed the management initiative to vet and review adjunct professor appointments regularly to ensure alignment, as positive.

NANO has expanded its engagement since 2017-18

Academia
  • 10 new agreements with universities other than the U of A (4 in ON, 3 in BC, 2 in AB, 1 in QC, and 1 foreign institution)
  • WIN-NRC Nanotechnology Joint Seed Funding Program
Other government departments (OGDs)
  • expanded to 4 DRDC projects
  • AAFC multi-year collaboration
  • NRCan collaboration via EME
Industry
  • multi-year collaborative research agreements with Hitachi and JEOL, which own 60% of the global electron microscopy market
Other NRC research centres (RCs)
  • working directly with 5 other RCs
  • microscopy network includes members from across 13 RCs

Strategic engagement

Finding

NANO has established a foundation for stronger partnerships, but engagement efforts could be more strategic. In terms of visibility, NANO has been inconsistent in its communications.

Source(s): Data review, document review, external and internal interviews, and peer review committee

Engagement activities could be more strategic

NANO participated in a number of conferences, consortiums, and virtual meetings. They launched the microscopy network and held workshops with clients/collaborators.

NANO is a member of the Canadian Centre for Electron Microscopy governance board, a key centre in Canada, and is promoting engagement with academia through adjunct professorships and new agreements. Stakeholder engagement is sometimes less direct as NANO works with other research centres who, in turn, work with various stakeholders across the country.

NANO is still developing its strategy to engage with the appropriate stakeholders in the most effective ways. Finding the proper mix of projects will have a major impact in determining NANO’s success.

In addition, NANO has yet to achieve the expected breadth of engagement due, in part, to their relationship with the U of A that uses a large portion of its overall resource pool and over-emphasizes academic projects. The PRC recognized the need for NANO to engage more with Canadian industries and explore multi-year, revenue-generating projects.

Need to increase visibility and outreach

NANO is not communicating their research expertise and platforms efficiently and will benefit from engaging with partners in more targeted ways (e.g. bringing industrial partners to their facilities to highlight their capabilities).

63% of clients and collaborators were somewhat aware of NANO’s capabilities and services.

Stakeholders consulted and the RCAB also identified issues with NANO’s visibility, in-person and online. NANO’s website is difficult to navigate and content is not up-to-date. There are limited examples of collaborative projects, contributions and achievements of teams. The signage at its facilities still identifies it as NINT.

Capabilities

NANO generally has the capacity, competencies, and facilities to achieve its current objectives. However, the research centre could be challenged moving forward due to limited revenues, budget challenges as well as the condition of facilities, some of which are in need of investment.

Capacity

Finding

Following the transition from NINT to NANO, the research centre had difficulty achieving the right mix of staff to align with its new strategic direction. To mitigate this, it has drawn on its connections to academia, supplementing staff with short-term, co-op students and research associates.

Source(s): Data review, document review, facilities review, external and internal interviews, and peer review committee

NANO is challenged in achieving the right mix of staff

Following its inception, NANO reorganized its teams and worked to optimize the allocation of staff to research areas and platforms according to their strategic plan. However, achieving this remains an ongoing process as NANO:

  • faces challenges attracting personnel to Alberta within the context of a competitive nanotechnology labour market
  • is not expecting much staff turnover in the near future, making it difficult to hire new staff in strategic areas
  • suffered significant turnover in leadership positions that was addressed by promoting from within the research centre
  • did not advance EDI goals on hiring women (only 2 of the 9 management and team lead positions are held by women)

NANO has leveraged capacity from universities

NANO has mitigated its capacity challenges by leveraging more capacity from academia. Firstly, NANO increased its reliance on short-term co-op students and research associates (up from 10 to 28). Secondly, NANO encourages its researchers to take on adjunct professorships at universities to access third-party funds and students (7 adjunct professorships at the U of A and 1 at the UBC as of 2021-22). Appointments are regularly vetted by management to ensure strategic alignment with the NANO mission.

To address leadership issues, the PRC recommended NANO establish or adjust processes to manage and promote its leaders. It viewed NRC efforts to increase diversity in STEM through national workshops and contests as positive steps.

Figure 14. Students and research associates supplement full-time research capacity

Note: Co-op students were employed by NANO for only part of the year. Also, research associates includes one post-doctoral fellow.

"Students and research associates supplement full-time research capacity "
Figure 14 – Text version

A clustered column chart illustrating number of researchers at the NANO Research Center from 2017-18 and 2020-2021.

  • During this time period the amount of students increased from 11 to 19 and the number of research associates increased from 0 to 9.
  • The amount of research officers decreased from 27 to 24 and the number of technical officers decreased from 23 to 22.
 

Competencies

Finding

NANO is working to stabilize teams so that they align with the research centre’s mission; however, more cross-team collaboration is needed to strengthen capabilities and provide greater value to clients and collaborators.

Source(s): Data review, external and internal interviews, and internal survey

The majority of staff (77%) feel that opportunities exist to increase engagement and/or collaboration across teams within NANO.

Over half of NANO staff (54%) engage across teams ‘often’, and another quarter ‘occasionally’ (27%). NINT-era staff are almost twice as likely to report cross-team collaboration (63% vs 35%).

NANO is building cross-team engagement and collaboration

NANO management indicates that their vision is to have cohesive teams working together. At the beginning of NANO, they established the teams as entities, and they are now working on cross-team collaboration. To support these efforts, NANO holds quarterly research days for ideation, brainstorming and presentation of new equipment.

Success will depend on teams coming together

The PRC recognized the good efforts within teams to put together specific platforms that are being used to engage with important stakeholders, which shows their willingness to adapt their projects toward the new priorities of the NANO mission. In light of its set capacity, the PRC suggested that achieving that mission will depend on greater cross-team collaboration. The PRC provided examples for greater collaboration including:

  • the DAM team’s work on tomography on biomedical samples, with Japanese collaborators, could benefit from more engagement with the BMN team
  • cross-team collaborations to make nanomaterials for sensors and other technology will better align the DAT and NDC teams’ work with the NANO mission
Figure 15. NANO cross-team engagement happened in a variety of ways

Source(s): Internal survey (n=43)

"NANO cross-team engagement happened in a variety of ways "
Figure 15 – Text version

A horizontal bar chart illustrating cross-team engagement in the NANO Research Center. The survey included 43 staff members and 6 indicators:

  • knowledge transfer/consultation (28% of staff reported engagement)
  • non-research support (26% of staff reported engagement)
  • collaborative work/projects (26% of staff reported engagement)
  • equipment/lab space sharing (14% of staff reported engagement)
  • staff/student sharing (7% of staff reported engagement)
  • training (5% of staff reported engagement)
 

Facilities

Finding

NANO has the facilities necessary to achieve its objectives, but strategies for co-investment, divestment or reorientation could be important to remain relevant and sustainable. NANO’s sustainability may be challenged, moving forward, if it does not further align its activities with its strategic direction.

Source(s): Data review, document review, facilities review, external and internal interviews, and peer review committee

NANO facilities meet Canadian needs

Most of NANO’s clients and stakeholders (68%) consider its facilities cutting-edge within Canada, however less than a third (29%) would say they stand out internationally. Of strategic value to the NRC are NANO’s microscopy facilities, which received significant investment during the NINT-era. The PRC affirmed that NANO has the facilities necessary to achieve its objectives but did find them to be inferior to Canadian universities such as McGill and McMaster.

Facilities underutilized, some unsupported

NANO facilities are under-utilized. NANO facilities, on average, have low utilization rates (less than 15%) despite high demand for similar industry and academic facilities. This was particularly acute for the NDC team’s characterization facilities.

Facility support is also an issue. Some equipment acquired during the NINT era is still available at NANO facilities, but there is a lack of staff to operate it. Moreover, some equipment service contracts were terminated to meet the NRC budget reduction; a development that concerned the PRC.

Cost-reduction strategies under consideration

NANO is working to reduce costs of its facilities and operations, in line with the current NRC budget reduction exercise. Interviewees suggested that NANO continue with cost-saving strategies such as cost-sharing with industry; subscription models with SMEs; and potentially decommissioning some facilities.

Without major capital, co-investment vital

Both the NRC Facilities Review and the PRC expressed concern over the ongoing sustainability of NANO’s microscopy facilities. A $5-$10M investment by 2024-25 is needed to exploit new opportunities for DAM. By comparison, the NDC’s deposition facilities require less than $1M by then. In the case of the characterization facilities, the Facilities Review exercise recommended NANO divest or re-orient these facilities towards new directions.

NANO management’s current strategy is to extend the capital budget by partnering with other organizations and using minor capital investments to make incremental improvements. In addition to leveraging existing partnerships, the PRC recommended spending less on internal projects and prioritizing external projects that may, in turn, enable co-investment. These partnerships are necessary as NANO facilities rank lower than other NRC facilities for major capital investment.

The NRC Facilities Review concluded in 2020-21. Internal perspectives were complemented by external peer reviews to assess the value of the NRC’s major facilities and provide recommendations for renewal. For NANO, the review focussed on its DAM.

Recommendations

Recommendations and supporting rationale

Research focus

Recommendation 1

NANO should further align its activities to its mission by:

  • narrowing the focus of its research platforms, services, and facility offerings
  • providing each research team/platform with a central focus and specific priorities, while promoting cross-team collaboration

Supporting rationale

NANO has made significant progress in reorganizing the NINT-era staff, facilities and projects and has achieved an impressive amount of collaborative research and networking initiatives. However, NANO experienced challenges in leadership and change management with staff and equipment transitioning from the NINT era to NANO. This included a lack of stable leadership and turnover in key lead and senior researchers.

The realignment of activities with the NRC’s mission and priorities is ongoing. The concept of R&D platforms is not fully understood nor recognized by staff. In many cases, the key scientific and technology leaders may be spread in too many areas. This can impede the clear focus needed to achieve high impact at the research center level.

NANO teams have integrated high quality standards in their research, highlighting that NANO has the training, capacity, and capability to achieve a strong level of excellence. However, there remains an uneven distribution in the level of effort across teams, which are still working in siloes in many cases. The PRC also recommended such improvements.

Project portfolio management

Recommendation 2

NANO should develop a balanced project portfolio management approach to ensure that projects are aligned with its mission and strategic objectives.

Supporting rationale

NANO has adopted a new project management system and selection criteria; however, justifications for choices and project alignment with NANO objectives are not clear and fully documented. In the context of the NI with the U of A, the project selection process, outside defined criteria, was limited. Using a project portfolio management approach, NANO should strengthen its project selection processes to better manage the flow and choice of projects towards cutting-edge nanoscience platforms and projects that align with NANO’s mission. This approach will enable the selection of lower TRL projects that can fill multiple and cross-team needs to help free resources and focus on external and high TRL projects. Moving towards a higher TRL focus would make the research centre more attractive to industry.

The process by which projects are undertaken and approved by NANO needs to be less opportunistic or based on team members “comfort zone” and purposefully embed cross-team or cross-functional collaborations to optimize innovation and project performance. NANO’s reliance on a limited number of multi-year revenue-generating projects with few OGDs and industries constitutes a considerable risk moving forward. The PRC also recommended such improvements.

Capacity

Recommendation 3

NANO should further leverage resources through partnerships and secure revenues from external collaborative research projects.

Supporting rationale

Financial resources available for R&D operations are limited and will further decline moving forward. NANO facilities are in need of investment, but may not be a priority given the strategic reinvestment needs in other NRC key facilities. To date, NANO has integrated budget reductions into facilities and operations through reduced technical service contracts, extended its capital budget by partnering, and used minor capital to make incremental improvements to the infrastructure. However, NANO should seek to secure revenue through larger collaborative research projects with Canadian industries and other federal government departments. There remain opportunities for NANO to leverage resources (talent, equipment, and funding) through partnerships beyond the U of A. The PRC also recommended such improvements.

Engagement

Recommendation 4

NANO should improve its outreach by engaging with external clients outside Alberta, to achieve a higher proportion of collaborative research projects with industry.

Supporting rationale

NANO did not reach its target of one third of operational budget dedicated to external research projects, and its client base remains concentrated in Alberta. Therefore, there is potential for NANO to further engage (outside Alberta) with Canadian industries, OGDs, and international clients. In particular, there are opportunities to engage more directly with clients and collaborators of other RCs through NANO’s involvement in the NRC challenge programs and by leveraging other RCs collaboration networks with various industry sectors and regions. As such, NANO should update its engagement plan including engaging via industry workshops and sectoral/regional industry associations. The PRC also recommended such improvements.

Awareness

Recommendation 5

NANO should increase the visibility and awareness of its research foci, capabilities and platforms.

Supporting rationale

Stakeholder awareness of NANO is low. For many, the NANO profile is still associated with microscopy expertise and NINT. Stakeholders, including other NRC RCs, are not aware of the research foci, capabilities and facilities of NANO in Edmonton. As a result, there are missed opportunities for NANO to engage inside and outside the NRC. It is recognized that the COVID pandemic hindered NANO’s ability to effectively communicate their new strategy and structures following the transition from NINT.

As such, there is a need to update the NANO website and improve communication material used by researchers and the client engagement team. Importantly, there are opportunities to better communicate NANO’s mission, teams and platforms, and showcase key collaborative research projects. Furthermore, signage on the NANO building and on the U of A campus still reference ‘NINT’, rather than ‘NANO’. This hinders NANO repositioning and branding efforts. The PRC and RCAB also recommended such improvements.

Performance Measurement

Recommendation 6

NANO should develop performance indicators to monitor achievement of its mission and strategic objectives, including the balance of its project portfolio.

Supporting rationale

NANO requires performance indicators to determine progress towards the achievement of NANO’s mission and priorities. Beyond academic metrics, indicators should demonstrate productivity of NANO; the balance of its project portfolio; and the impact of work with internal and external clients and collaborators.

While the performance information and data available for this evaluation was better in comparison to the previous evaluation of NINT, there is an opportunity for NANO to leverage the evaluation findings to develop and track specific RC-relevant performance indicators. The PRC also recommended such improvements.

Management Response and Action Plan

Management Response and Action Plan
Recommendation Risk‑level Management response Measure of achievements Proposed person(s) responsible Expected date of completion
1. NANO should further align its activities to its mission by:
  • narrowing the focus of its research platforms, services and facility offerings
  • providing each research team/platform with a central focus and specific priorities, while promoting cross-team collaboration
High

Response: Accepted

Action 1: The RC will develop a strategy to identify and/or reinforce priority areas for each Team, and to align resources to priority. The RC also will work with its new facilities team to optimize utilization of required equipment and to identify equipment for decommissioning with attention to relevance, usage and cost.

Action 2: In addition, the RC will develop a strategy to promote cross-team collaboration while also maintaining focal areas by incentivising cross-team projects that are aligned.

Action 1:

  • Strategy that includes a narrowed research focus for platforms, aligns activities to NANO’s mission, brings clarity to required facilities and provides a central focus for each team developed.

Action 2:

  • Strategy to promote cross-team collaboration developed.
Director, Research and Development with support from Director of Operations and Project Management Office

Action 1: December 2022

Action 2: April 2023

2. NANO should develop a balanced project portfolio management approach to ensure that projects are aligned with its mission and strategic objectives.

High

Response: Accepted

Action 1: The RC will develop a re-vamped project selection framework and portfolio management approach to ensure alignment of projects to its mission and strategic objectives.

Action 2: The RC will develop a plan for sharing of resources across teams for targeted and aligned projects, and these will be a priority. Resources will be redeployed as needed in order to grow external collaborations and services.

Action 1: A new project selection framework and project portfolio management approach developed.

Action 2: A new resource allocation plan developed.

Director General, with support from Director, of Research and Development, Director of Operations, and Project Management Office

Action 1: December 2022

Action 2: April 2023

3. NANO should further leverage resources through partnerships and secure revenues from external collaborative research projects. Moderate

Response: Accepted

Action 1: The RC will identify receptors and partners for its platforms and for revenue-generating strategic research projects through direct outreach to industry, other NRC RCs and OGDs, including building on relationships with DRDC, ECCC, NRCan, AAFC, and others.

Action 2: The RC will also work with industry users to define models for access and utilization of clean room and other facilities that will support their needs and provide revenue for NANO.

Action 1: Strategy to increase revenue-generating project agreements developed.

Action 2: Strategy to increase client agreements for facility access and equipment utilization, including clean room and microscopy suite developed.

Director General, Director, Research and Development, and Business Management Services

Action 1: December 2022

Action 2: December 2022

4. NANO should improve its outreach by engaging with external clients outside Alberta, to achieve a higher proportion of collaborative research projects with industry. Moderate

Response: Accepted

Action 1: The RC will develop a strategic plan for engaging new partners outside Alberta while maintaining core aligned activities and managing capacity.

Action 2: The RC will engage with industry through a variety of events to promote its capabilities and to explore collaborative research projects opportunities with industry. As well, The RC will continue to engage with other RCs across the country in order to engage more directly their networks of industry partners, clients, and collaborators. Examples include delivery and/or participation to workshops, open houses, industry days, and trade shows.

Action 1: Strategic plan for engaging partners outside of Alberta developed.

Action 2: Workshops and networking events with industry and other key stakeholders delivered and/or attended, and coordinated with engagement plan.

Director of Research and Development with support from Business Advisor and Communications

Action 1: November 2022

Action 2: October 2023

5. NANO should increase the visibility and awareness of its research foci, capabilities, and platforms. Low

Response: Accepted

Action 1: The RC will continue to work with NRC Communications in order to promote its work. These efforts will include press releases, social media postings, and development of a range of collateral to establish a brand.

Action 2: NANO will work to have signage updated in collaboration with Real Property Planning and Management (RPPM).

Action 1: Communication plan to increase visibility and awareness of NANO developed (e.g. social media/web-site updates, videos and virtual tour content).

Action 2: Signage updated on the building as well as around the U of A campus.

Director General with support from Director, Research and Development, Director of Operations, Business Advisor, Real Property Planning and Management, and Communications

Action 1: December 2022

Action 2: September 2022

6. NANO should develop performance indicators to monitor achievement of its mission and strategic objectives, including the balance of its project portfolio. Low

Response: Accepted

Action 1: Relates to Recommendation 1 and 2, and developing performance metrics will be essential for management and Team Leads to properly monitor progress toward the achievement of NANO’s mission and strategic objectives. The RC will work with Performance Measurement & Accountability Reporting (PMAR) to finalize its Management Tool Kit (corporate and specific indicators and logic model), adding indicators referring to the portfolio management approach.

Action 2: The RC will develop indicators through consultation with NANO teams (to assure alignment efforts are effective), as well as PMAR.

Action 1: Management Tool Kit finalized including specific key indicators to monitor achievement of NANO’s mission (e.g. alignment/balance of project portfolio, research excellence).

Action 2: Performance indicators are developed and implemented in CTEs and Project Management Office documentation.

Director General with support from Director of Operations, and Project Management Office

Action 1: April 2023

Action 2: April 2023

 

Appendices

Appendix A – Methodology

Document review

Internal and external documents were reviewed to provide context and to complement other lines of evidence in assessing relevance and performance. Internal documents included operational, strategic and stakeholder engagement plans, portfolio monthly reports, advisory board terms of reference, meeting minutes and presentations, facilities reviews, project selection and prioritization documents, initiative-specific documentation (i.e. NI, Microscopy network).

External documents included documents related to government priorities (ex: NRC strategic plan and mandate letters departmental plans), benchmarking studies on international comparators by area of research and an environmental scan of market data from 5 capability areas, both prepared by the NRC’s Library and Information Management Services (NRC LIMS), and documents related to federal government priorities.

Data review

The research centre’s administrative and performance data for 2015-16 to 2020-21 were reviewed to provide information on program inputs (i.e. resources), outputs (e.g. projects and technical services, platforms, IP) and client engagement (e.g. research, facilities and bilateral agreements, clients and collaborators lists).

This included key performance indicators, financial data and minor capital allocations, decisions, human resource data, project data and client data.

Internal and external interviews

Interviews were conducted to collect information such as personal experiences, opinions and expert knowledge related to the research centre’s relevance, engagement, resources, facilities, scientific excellence and achievement of expected results.

A total of 51 stakeholders were interviewed for this evaluation:

  • 16 (or 31%) internal staff (management team, team leads, research officers and support staff)
  • 22 (or 43%) external clients and collaborators
  • 13 (or 25%) stakeholders (non-clients)

External interviewees included representatives from Canadian universities and other research organizations (n=13); Canadian industry and associations (n=9); NRC research centres (n=4); other federal and provincial government departments (n=5); and foreign universities, industry and national laboratory (n=4).

This count includes the interviews conducted in the contexts of project case studies (for details see the Project case studies section).

Information gathered through the interviews provided contextual information and was used in conjunction with the other lines of evidence.

In addition to sectors, regions and categories of NANO stakeholder representation, efforts were made to include diversity of perspective. As such, 27% of interviewees and survey respondents were women and 73% were men.

Internal staff survey

A total of 80 Nanotechnology research centre staff were invited to compete an online survey which included questions related to the transition, engagement capabilities and scientific excellence of the research centre activities.

48 internal staff responded for a response rate of 60%.

Client hybrid survey

Online survey: A total of 48 external clients and collaborators were invited to complete an online survey or to answer interview questions on the transition, relevance, engagement, capabilities, scientific excellence and impact of the research centres activities.

19 external clients and collaborators responded for a response rate of 41%.

Interview data: An additional 34 external interviewees were included in survey responses to supplement specific close-ended questions.

A total of 53 clients and collaborators were included in this hybrid survey, covering 81 % of NANO client base. This breakdown of survey respondents was 40% from Canadian industry 38% from academia/other, 10% from the NRC, 8% from other government departments, 2% from foreign industry and 2% from foreign national laboratory.

Bibliometric study

The Intelligence and Analytics Group of NRC LIMS conducted a bibliometric analysis of peer-reviewed publications affiliated with the research centre and indexed in Scopus for the period of 2015-16 to 2020-21. This analysis was used to answer questions on scientific excellence (publications output and relative citations overall, and by area of research) and engagement (co-authorship/scientific collaboration).

Authorship of NANO-NINT publications classified into 2 categories:

  • NANO (i.e. publications authored by NANO researchers)
  • NANO+NINT (i.e. publications authored by either NANO or NINT researchers)

Project case studies

Project case studies were conducted on 7 NANO projects completed between 2018-19 to 2020-21 to illustrate scientific and technological achievements as well as outcomes for participating organisations and impacts. These projects were chosen to be representative of NANO's research thrusts, project types, client types and project sizes. Projects studied can be grouped in 3 main categories:

  • capability development projects (2 projects);
  • round 1 Nanotechnology Initiative projects (2 projects), and;
  • strategic collaborative research projects with industry and OGD clients (3 projects)

This case study approach included a review of project documents as well interviews with the NANO project lead and at least 1 representative from the client or collaborator organizations. In total, 14 interviews were conducted – 6 with clients and 8 with project leads. These project-level consultations are included in the interview distribution presented under “Internal and external interviews”.

Appendix B - Limitations and mitigations

Although the evaluation encountered some challenges, methodological limitations were mitigated, where possible, through the use of multiple lines of evidence and the triangulation of data. This approach was taken in order to establish the reliability and validity of the findings and to ensure that conclusions and recommendations were based on objective and documented evidence. Details on limitations and their associated mitigation strategies are described below.

Limitations Mitigations

Lack of updated logic model, NANO-specific performance indicators

At the outset of the evaluation NANO did have an updated strategic plan, but their logic model and RC-specific performance indicators were not available (i.e., its management toolkit). Consequently, the expected outcomes in its previous logic model did not align with its current plan and mission. Also, while the plan did include some performance information, key NANO-specific performance data were not readily available.

  • Without a current logic model aligned to its mission, NANO was assessed against the NRC’s overall goals, namely contributions to government policy solutions, business innovation, and environmental, health, and social impacts.
  • Without NANO-specific performance indicators, OAE collected primary data (client and stakeholder interviews and surveys), accessed existing secondary data (NRC-wide key performance indicators including client counts, publications, and revenue) and commissioned a bibliometric study from NRC-LIMS.

Limited project performance data

The project lists and proposals initially provided by NANO to OAE included only limited accounts of outputs and outcomes. While the quantity and quality of information was greater than that provided by NINT in 2015-16, it was challenging to assess projects’ alignment with NANO’s mission and achievement indented impacts.

OAE and NANO worked together to enable both the PRC and OAE to fully assess NANO’s scientific excellence and impact:

  • OAE conducted detailed reviews of 7 projects, including document reviews, and interviews with both NANO staff and its clients and collaborators.
  • Additionally, NANO provided detailed profiles of 9 projects including their rationale, needs, results and impacts.
  • The PRC sessions allocated significant time for reviewers to engage NANO team leads, including presentations, breakout rooms, and multi-day Q & A sessions.

Limited data on contributions to NRC programs

It was not possible to quantify the level of effort NANO expended on Challenge or Supercluster Programs coordinated by other RCs as the NRC stopped tracking inter-RC resource sharing during the evaluation period.

To validate and assess NANO’s contributions to NRC programs, OAE:

  • interviewed a sample of staff from 4 other RCs (AEP, EME, HAA, and METRO) who could provide an overview of NANO’s contributions.
  • reviewed program documentation that referenced NANO’s involvement.

Appendix C - Peer review committee

A peer review committee (PRC) was convened to assess the research centre’s relevance, engagement and scientific excellence. The committee was composed of 6 members with expertise in electron and atom probe microscopy; nanomaterials and microsystems; nanosensors; biomedical nanotechnologies; nanoscience; and computer modelling.

PRC Members included national and international representatives from academia, research organizations and industry. To ensure objectivity and avoid conflicts of interest, members signed a confidentiality and conflict of interest agreement.

PRC members examined key research centre documents, the preliminary evaluation findings and facilities reviews prior to participating in 4 virtual review sessions. Virtual sessions included presentations and poster sessions delivered by the research centre, interaction with NANO’s research team and management, and NRC senior executive.

Based on the documents reviewed and the virtual sessions, the PRC produced a report that responded to each of the areas under assessment. This report was used as a line of evidence in the overall evaluation findings.

Dr. Patricia Nieva (Committee Chair)
Professor, Deputy Chair, Mechanical and Mechatronics Engineering
Director, Sensors and Integrated Microsystems Laboratory
University of Waterloo

Dr. Mark Hybertsen
Senior Scientist & Group Leader, Theory and Computation
Center for Functional Nanomaterials
Brookhaven National Laboratory

Dr. Karren Moore
Director, Center for Nanophase Materials Sciences
Oak Ridge National Laboratory

Dr. Carolyn Moorlag
Director, CAMDT and FAST Research
Sheridan College
Chair, Research Centre Advisory Board
NANO, NRC

Dr. Steve Petrone
Founder, CEO & CTO
Quantiam Technologies Inc.

Dr. Anjali Sharma
Research Faculty, Center for Nanomedicine
Johns Hopkins University School of Medicine