Evaluation of NRC's Security and Disruptive Technologies Research Centre - Evaluation report summary

 

Alternative format

PDF version (404 KB)

January 8th, 2018

Prepared by:
Office of Audit and Evaluation
National Research Council Canada

Approval:
This report was approved by NRC's President on January 8th, 2018

Program description

Created in 2012, the Security and Disruptive Technologies (SDTech) Research Centre's primary goal is to catalyze Canadian global leadership in select longer-range emerging and disruptive technologies of strategic importance to Canada's economy. SDTech's mandate aims to advance technology from low to medium technology readiness levels (TRL) to de-risk early stage development of strategic technology platforms and transfer technologies to collaborative partners who lead subsequent higher-TRL programs.

To fulfill its objectives, SDTech works with other NRC research centres and external partners and clients, such as regional Research Technology Organizations (RTOs), other government departments (OGDs), academic institutions and industry, to identify and develop application-focused programs that accelerate the development of pervasive technology platforms most relevant to Canada. The markets currently targeted by the SDTech Research Centre include information and communication technologies (ICT), energy and environment, and defence and security.

Financial resources

Revenue in millions $
Fiscal year Technical Services Strategic Research
2016-17 0.78 1.63
2015-16 0.84 1.44
2014-15 0.49 1.39
2013-14 0.58 1.59
2012-13 0.78 1.85
 

Between FY 2012-13 and 2016-17, revenues generated were around $11.37 M (70% strategic research services) and expenditures were around $50.6 M (88% allocated to strategic research projects).

Scope and methodology

The evaluation of the SDTech Research Centre and its two programs (QPSS and SMT) covered the period from fiscal year (FY) 2012-13 to 2016-17 inclusive. The evaluation was carried out in accordance with the NRC's approved evaluation plan and TBS policies. The Research Centre and its programs had not been previously evaluated.

Data was collected by NRC's independent evaluation team, supported by an external consultant. The evaluation employed qualitative and quantitative research methods (document and data review, interviews, market assessment, peer review).

Challenges and mitigation strategies

  • Outdated bibliometric data. Additional information on key researcher accomplishments (awards, prizes) was provided to committee members.
  • No comparisons of program performance with similar organizations during data collection.The peer review was relied upon to make comparisons.
  • Unconscious biases in peer review committees. Members were drawn from various organizations and regions and included female representation.
  • Limited external interviews. Information was triangulated across several lines of evidence.
  • Short site visit (1.5 days). Committee members were provided materials and questions were answered prior to the visit.

Overall evaluation findings

Need for R&D activities

QPSS: addresses a need in the area of quantum photonics research and development, as quantum technologies are expected to have a significant impact on industries in the future. SMT: addresses stakeholder needs in the area of advanced materials research and development (R&D), but not to its full potential.

SMT: addresses stakeholder needs in the area of advanced materials research and development (R&D), but not to its full potential.

Scientific excellence

SDTech: researchers perform above the world average in terms of the relative impact of their published work, and are recognized nationally and internationally for their research excellence. Success has been achieved in translating research into promising technologies.

QPSS: conducts world-leading research in the area of attosecond science. Research conducted in the fields of molecular photonics, quantum dot research and microscopy are considered outstanding.

SMT: conducts world-leading research in the development and application of boron nitride nanotubes (BNNTs); however, these efforts have not been as visible within the scientific community in the past few years.

Facilities excellence

QPSS: for the most part, quantum photonics research installations are world-class. The lack of access to high performance computing equipment was identified as a weakness.

SMT: the nanotechnology facilities were considered comparable to those of other national institutions working in this field of research, with the exception of the production of BNNTs which was deemed unique. The absence of a scale-up facility is restricting the ability of the program to pursue technology transfer opportunities.

Client and stakeholder engagement and ecosystem development

SDTech: overall, clients and stakeholder value their relationships with SDTech and reported good alignment with their long term goals, however immediate needs and expectations with respect to commercialization have not always been met.

QPSS: the level of engagement with quantum stakeholders in Canada was found to be above average in terms of the breadth of stakeholders engaged. Concerns were raised regarding the level of engagement with entities that can help bridge the gap between scientific discovery and commercialization. SMT: while expected to increasingly engage industry, the program continues to rely on DRDC as its primary client. The program has successfully developed the Security Materials Technology Roadmap (SMTRM), however, there is limited evidence of implementation beyond the sharing of information.

SMT: while expected to increasingly engage industry, the program continues to rely on DRDC as its primary client. The program has successfully developed the Security Materials Technology Roadmap (SMTRM), however, there is limited evidence of implementation beyond the sharing of information.

Recommendations and management response

QPSS - Recommendation 1

Develop a strategy to ensure the world-leading status of the fundamental research carried out within the program is maintained

Management Response: Recommendation accepted.

SDTech will develop a strategy, including:

A1. Exploring opportunities to establish an NRC Collaboration Centre with the Perimeter Institute in Waterloo.

A2. Retaining existing adjunct professorships and seeking to obtain additional adjunct professorships at Canadian universities.

B. Reviewing collaborative research models such as JILA.

C. Exploring avenues to access high performance computing (HPC) and selecting one or more of these avenues as appropriate.

D. Formulating a succession plan to address pending retirements of key researchers.

QPSS - Recommendation 2:

Develop a strategy and execution plan for the Quantum Security Technology Access Centre (QSTAC) in order to maximize the impacts of the joint collaboration between the NRC and other government departments (OGDs), industry and academia.

Management Response: Recommendation accepted.

SDTech will develop a strategy and execution plan for QSTAC as part of the proposal for the next iteration of QPSS (the Quantum Foundational Program).

QPSS - Recommendation 3:

Develop a strategy to further advance technologies in the security, resource and environmental sensing areas.

Management Response: Recommendation accepted.

The Quantum Foundational Program proposal will include a strategy for technology advancement.

SMT - Recommendation 4:

Place more emphasis on the fundamental research being conducted within the program as a foundational requirement for advanced materials development.

Management Response: Recommendation accepted.

The next iteration of SMT will be the Adaptive and Intelligent Materials Foundational Program (AIM) which will have a greater focus on lower TRL work.

A1. SDTech will retain existing adjunct professorships and seek to obtain additional adjunct professorships at Canadian universities.

A2. SDTech will have graduate students working with staff who are adjunct.

B. As indicated in Recommendation 1C: Looking at potential avenues to access HPC and selecting one or more of these avenues as appropriate.

C. Continue collaboration on BNNT for single photon sources. Opportunities will be explored as part of the yearly program technology project proposals, held in Q1 of each year.

SMT - Recommendation 5:

Refocus technology transfer efforts, given available resources and the program's lifecycle.

Management Response: Recommendation accepted.

A. Explore the possibility of co-funding the scale-up facility with DRDC and the instrument suppliers. If co-funding is not feasible, SMT will revise the program objectives and technology transfer strategy.

B1. Discussions with Automotive and Surface Transportation (AST) and Aerospace (AERO) Research Centres to determine pertinent technologies to transfer.

B2. Continue to explore innovative approaches for working with industry to further technology development and transfer.

B3. Review the type and level of specialized organizational support required for technology transfer and business development and establish a renewed strategy.

SMT - Recommendation 6:

Expand the program's client base beyond DRDC, particularly with the end-user community, where appropriate.

Management Response: Recommendation accepted.

A1. Explore opportunities to expand business with existing customers (e.g., Dew, Nortrax and Mawashi) and continue its expanded business activities with Tekna.

A2. Develop additional engagements with SMTRM participants across the advanced materials value chain, using workshops and direct engagements with companies.

B. Explore alternate applications and assess how technologies developed can be used beyond the defense industry and approach new potential clients.