Evaluation of NRC's Security and Disruptive Technologies Research Centre - Final Report, January 8th, 2018

 

Alternative format

PDF version (864 KB)

Prepared by: Office of Audit and Evaluation, National Research Council Canada

Approval: This report was approved by NRC 's President on January 8th, 2018

Acronyms and Abbreviations

AEP Advanced Electronics and Photonics
AV Armoured vehicle
BNNT Boron nitride nanotube
CIFAR Canadian Institute for Advanced Research
CNT Carbon nanotube
CSE Communications Security Establishment
DND Department of National Defence
DRDC Defence Research and Development Canada
FY Fiscal Year
HPC High performance computing
ICT Information and communication technologies
IP Intellectual property
JASLab Joint Attosecond Science Laboratory
NRC National Research Council
NSERC Natural Sciences and Engineering Research Council
OAE Office of Audit and Evaluation
OGD Other government department
PE Printable Electronics
PPE Personal protective equipment
PRC Peer review committee
QPSS Quantum Photonic Sensing and Security
QSTAC Quantum Security Technology Access Centre
R&D Research and development
SDTech Security and Disruptive Technologies
SMT Security Materials Technology
SMTRM Security Materials Technology roadmap
TRL Technology readiness level

Executive Summary

This report presents the results of the evaluation of the National Research Council's (NRC) Security and Disruptive Technologies (SDTech) Research CentreFootnote 1. Created in 2012, SDTech 's primary goal is to "catalyze Canadian global leadership in select longer-range emerging and disruptive technologies of strategic importance to Canada's economy"Footnote 2. During the period covered by the evaluation (fiscal years 2012-13 to 2016-17), SDTech included two NRC programs: Quantum Photonic Sensing & Security (QPSS) and Security Materials Technology (SMT). This evaluation assessed the relevance and performance of the SDTech Research Centre as well as its two programs.

The key findings for each evaluation issue have been summarized and are presented below. Where applicable, findings are presented for SDTech as a whole, as well as for the QPSS and SMT programs separately.

Key Findings: Need for R&D activities

  • The QPSS program is addressing a need in the area of quantum photonics research and development, as quantum technologies are expected to have a significant impact on industries in the future.
  • The SMT program is addressing stakeholder needs in the area of advanced materials research and development (R&D), but not to its full potential.

Key Findings: Scientific excellence

  • SDTech researchers perform above the world average in terms of the relative impact of their published work, and are recognized nationally and internationally for their research excellence.
  • SDTech has been successful in translating its research into promising technologies.
  • The QPSS program is conducting world-leading research in the area of attosecond science. In addition, the research conducted in the fields of molecular photonics, quantum dot research and microscopy are considered outstanding.
  • The SMT Program is conducting world-leading research in the development and application of boron nitride nanotubes (BNNTs). However, these efforts have not been as visible within the scientific community in the past few years.

Key Findings: Facilities excellence

  • For the most part, the QPSS program's quantum photonics research installations are world-class. The lack of access to high performance computing equipment was identified as a weakness.
  • The nanotechnology facilities used by the SMT program were found to be comparable to those of other national institutions working in this field of research, with the exception of the production of BNNT s which was considered unique.
  • The absence of a scale-up facility is restricting the ability of the SMT program to pursue technology transfer opportunities.

Key Findings: Client and stakeholder engagement and ecosystem development

  • While, overall, clients and stakeholder value their relationships with SDTech and reported good alignment with their long term goals, immediate needs and expectations with respect to commercialization have not always been met.
  • The level of engagement with quantum stakeholders in Canada was found to be above average in terms of the breadth of stakeholders engaged.
  • Concerns were raised regarding the QPSS program's level of engagement with entities that can help bridge the gap between scientific discovery and commercialization.
  • While over time the SMT program was expected to increasingly engage industry, the program has not yet achieved this outcome and continues to rely on DRDC as its primary client.
  • The SMT program has been successful in developing the Security Materials Technology Roadmap (SMTRM), however, there is limited evidence of implementation beyond the sharing of information.

Conclusions and Recommendations

The evaluation of SDTech revealed that, overall, the Research Centre is, through its two programs, addressing client needs and achieving scientific impacts. Efforts to engage key stakeholders are underway through the Quantum Canada and SMTRM initiatives. Technology development is in the early stages and presents some challenges moving forward, which is not surprising, given that the Research Centre is engaged in activities at low TRL levels.

With respect to each of the programs, the peer review committee provided the following recommendations which have been abbreviated for this executive summary (the full text of the recommendations can be found in Section 4 of the report). In each case, the recommendation is followed by the management response.

QPSS Program Recommendations

Recommendation 1:

Develop a strategy to ensure the world-leading status of the fundamental research carried out within the program is maintained.

Management Response: Recommendation accepted

SDTech will develop a strategy, including:

A1. Exploring opportunities to establish an NRC Collaboration Centre with the Perimeter Institute in Waterloo.

A2. Retaining existing adjunct professorships and seeking to obtain additional adjunct professorships at Canadian universities.

B. Reviewing collaborative research models such as JILA.

C. Exploring avenues to access high performance computing (HPC) and selecting one or more of these avenues as appropriate.

D. Formulating a succession plan to address pending retirements of key researchers.

Recommendation 2:

Develop a strategy and execution plan for the Quantum Security Technology Access Centre ( QSTAC ) in order to maximize the impacts of the joint collaboration between the NRC and other government departments (OGDs), industry and academia.

Management Response: Recommendation accepted

SDTech will develop a strategy and execution plan for QSTAC as part of the proposal for the next iteration of QPSS (the Quantum Foundational Program).

Recommendation 3:

Develop a strategy to further advance technologies in the security, resource and environmental sensing areas.

Management Response: Recommendation accepted

The Quantum Foundational Program proposal will include a strategy for technology advancement.

SMT Program Recommendations

Recommendation 4:

Place more emphasis on the fundamental research being conducted within the program as a foundational requirement for advanced materials development.

Management Response: Recommendation accepted

The next iteration of SMT will be the Adaptive and Intelligent Materials Foundational Program (AIM) which will have a greater focus on lower TRL work.

A1. SDTech will retain existing adjunct professorships and seek to obtain additional adjunct professorships at Canadian universities.

A2. SDTech will have graduate students working with staff who are adjunct.

B. As indicated in Recommendation 1C: Looking at potential avenues to access HPC and selecting one or more of these avenues as appropriate.

C. Continue collaboration on BNNT for single photon sources. Opportunities will be explored as part of the yearly program technology project proposals, held in Q1 of each year.

Recommendation 5:

Refocus technology transfer efforts, given available resources and the program's lifecycle.

Management Response: Recommendation accepted

A. Explore the possibility of co-funding the scale-up facility with DRDC and the instrument suppliers. If co-funding is not feasible, SMT will revise the program objectives and technology transfer strategy.

B1. Discussions with Automotive and Surface Transportation (AST) and Aerospace (AERO) Research Centres to determine pertinent technologies to transfer.

B2. Continue to explore innovative approaches for working with industry to further technology development and transfer.

B3. Review the type and level of specialized organizational support required for technology transfer and business development and establish a renewed strategy.

Recommendation 6:

Expand the program's client base beyond DRDC , particularly with the end-user community, where appropriate.

Management Response: Recommendation accepted

A1. Explore opportunities to expand business with existing customers (e.g., Dew, Nortrax and Mawashi) and continue its expanded business activities with Tekna.

A2. Develop additional engagements with SMTRM participants across the advanced materials value chain, using workshops and direct engagements with companies.

B. Explore alternate applications and assess how technologies developed can be used beyond the defense industry and approach new potential clients

 

Introduction

This report presents the results of the evaluation of the National Research Council's (NRC) Security and Disruptive Technologies ( SDTech ) Research CentreFootnote 3, including reviews of each of its two programs: Quantum Photonic Sensing & Security (QPSS) and Security Materials Technology (SMT). SDTech was selected for evaluation based on consultations with NRC Senior Management and the work was carried out in accordance with NRC 's approved evaluation plan. The Research Centre had not previously been the subject of an evaluation since it was launched in 2012. The evaluation covers the period from fiscal year 2012-13 to 2016-17.

The evaluation was led by an independent evaluation team from NRC 's Office of Audit and Evaluation (OAE), supported by a consultant. The evaluation methodology included a limited number of lines of evidence in keeping with the relative low level of riskFootnote 4 assigned to the evaluation during the planning phase. The methodology included the following lines of evidence:

  • Document review
  • Analysis of financial, administrative and performance data
  • Key informant interviews (internal staff/management n=19, external partners/stakeholders n=15)
  • Market assessment

Information gathered through these lines of evidence was provided to a peer review committee (PRC) comprised of seven experts from Canada and the United States with expertise in SDTech 's main research areas. The PRC members reviewed the information, prepared a preliminary assessment, and attended a site visit at the National Research Council in Ottawa. The list of committee members and the site visit agenda can be found in Appendix A. A more detailed description of the evaluation methodology and its limitations is provided in Appendix B.

The evaluation report is organized as follows:

  • Section 2 provides a profile of SDTech and its programs
  • Section 3 presents the evaluation study's findings organized by theme
  • Section 4 presents the overall conclusion and recommendations
  • Section 5 lays out the management response to these recommendations and the actions that will result

2.0 Profile of SDTech

2.1 Research Centre and Program Structure

Created in 2012, the Security and Disruptive Technologies (SDTech) Research Centre's primary goal is to "catalyze Canadian global leadership in select longer-range emerging and disruptive technologies of strategic importance to Canada's economy”Footnote 5. SDTech 's mandate aims to advance technology from low to medium technology readiness levels (TRL) to de-risk early stage development of strategic technology platforms and transfer technologies to collaborative partners who lead subsequent higher- TRL programs.

To fulfill its objectives, SDTech works with other NRC research centres and external partners and clients, such as regional Research Technology Organizations (RTOs), other government departments (OGDs), academic institutions and industry, to identify and develop application-focused programs that accelerate the development of pervasive technology platforms most relevant to Canada. The markets currently targeted by the SDTech Research Centre include information and communication technologies (ICT), energy and environment, and defence and security. A logic model which outlines the Research Centre's activities and intended outcomes is provided in Appendix C.

SDTech , like other NRC Research Centres, is responsible for designing and sustaining technical competencies in people and facilities which are then deployed through NRC programs to achieve results. While programs are time bound and have a sharp strategic focus, Research Centres have a longer lifespan and only come to an end when/if there is a fundamental shift in the NRC 's strategy business goals. During the timeframe of the evaluation, SDTech operated two programs, each of which carried out specific activities in different areas of research. SDTech also supported the materials thrust of the Printable Electronics (PE) Program, which is operated by the Advanced Electronics and Photonics (AEP) Research CentreFootnote 6.

2.1.1 Quantum Photonic Sensing and Security (QPSS) Program

Program objectives and activities

Launched in 2014, the QPSS Program is a seven year program which aims to "enable the Canadian photonics industry to develop a dominant global market share in select market segments by developing innovative quantum solutions to photonics-based cyber security, natural resource measurement, and security measurement challenges"Footnote 7. The primary focus of the program is on the development of a quantum photonics platform, including low- TRL technologies in quantum photonics sensing, sources and materials, to support higher- TRL technology development in sensing and security applications. More specifically, the program's objective is to support the development of technologies that directly benefit Canadian industry in the following three areas:

  1. Cyber security for the ICT sector: development of new systems for ultra-secure quantum cybersecurity networks for communication encryption and information processing for government and financial institutions.
  2. Sensing for the defence and security sector: development of fibre-based and non-linear spectroscopy technologies for various defence, military and security applications, such as rapid chemical-specific detection of hazards (e.g., explosives and fissile materials).
  3. Quantum molecular imaging for the energy and environment sector (e.g., mining, oil and gas): development of fiber optic sensing technologies for extraction and processing of natural resources.

In addition to research and development (R&D) activities, the QPSS program is involved, along with key stakeholders from academia, industry and government, in the development of a national quantum strategy ("Quantum Canada" Strategy). The primary aim of the strategy is to "provide visible focus for Canada's national interests in quantum, advance research investment activity in the sector, and ensure that Canada's present-day advantage in quantum technologies is maintained and expanded for long-term economic prosperity"Footnote 8.

2.1.2 Security Materials Technology (SMT) Program

Program objectives and activities

The SMT Program was launched in 2013 and is jointly led by the NRC and Defence Research and Development Canada (DRDC)Footnote 9. The program was created to support the development of high-performance, cost-effective, and lightweight blast and ballistic protection systems, with a specific focus on advanced personal protective equipment (PPE) and armoured vehicle (AV) products. The objective of the program is to develop and transfer to industry cost-effective materials and armour systems with mass-efficiencies (i.e., protective level per unit mass) at least 25% better than those of existing systems.

Program R&D activities are divided into four thrusts which aim to support partners operating in the defence and industry sector, specifically in the security materials industry. Developments under the first thrust feed into the second thrust, which subsequently feeds thrusts three and four:

Thrust 1 – Advanced materials technology platform development: development and validation of the transformational potential of materials and derived technologies; demonstration of pilot scale production to validate whether laboratory advances can be converted to larger scale manufacturing. The primary focus under Thrust 1 is carbon nanotube (CNT) and boron nitride nanotube (BNNT) production.

Thrust 2 – Engineered materials products: development of high-performance and high-complexity materials for the development of PPE and AV armour structures and systems.

Thrust 3 – PPE products: development of new, high performance materials, material architectures and manufacturing technologies for PPE applications (e.g., ballistic inserts, vests, visors and combat helmets).

Thrust 4 – Vehicle armour products: development of passive composite armour that can be added to the side of existing Canadian military or civilian armoured vehicles, and passive transparent armour that can be used to replace existing transparent armour on military vehicles (e.g., windshields, side windows, etc.).

SMT program outreach activities have focused on the development of the Canadian Security Materials Technologies Roadmap (SMTRM), in collaboration with DRDC and other industry partners. The purpose of the SMTRM is to "accelerate innovation, competitiveness, and productivity in Canada's security materials technology sector"Footnote 10.

2.1.3 Materials Thrust of the Printable Electronics (PE) Program

The Printable Electronics Flagship Program was launched in early 2012 to facilitate and promote the early adoption of emerging PE technologies among Canadian industry stakeholders. One of the program's main activities has been collaboration with leading industry stakeholders involved in PE , mainly through NRC 's Printable Electronics Consortium, to conduct collaborative strategic R&D to develop PE technologies. More specifically, the PE program is conducting R&D in three areas:

  • Functional materials: Conductive and semiconducting ink, scalable processes
  • Functional devices: Logic circuits, memory, conductors, radio-frequency identification (RFID) and near-field communication (NFC)
  • Functional imprinting: innovative interactive optical features for security printing

The program's initial focus was to develop PE solutions for the packaging, commercial and security printing industries, however, following a recalibration process in 2015, the program is now targeting five market verticals: wearable technology, packaging, advanced materials, security printing, and smart buildings/construction.

SDTech personnel have contributed to PE 's R&D activities in advanced materials, specifically functional inks, which aim to improve the performance and manufacturability of conductive, semi-conductive, dielectric and sensing inks. Two technologies are currently being developed by the PE program in the area of functional inks: single walled carbon nanotube (SWCNT) inks and devices and conductive ink platform for printable electronics.

2.2 Research Centre Resources

2.2.1 Human Resources

Between 2013 and 2017, the number of R&D personnelFootnote 11 within SDTech remained stable at around 65, over two thirds of which have obtained a doctorate. Overall, the Research Centre employs approximately 83Footnote 12 staff.

SDTech personnel are divided into seven teams, each led by a team leader. The teams are deployed to support Research Centre programs and the projects within those programs. Areas of specialization are as follows:

  • Attosecond Science
  • Fibre Photonics
  • Nanomaterials Optical and Electronic
  • Nanocomposites
  • Quantum Theory
  • Molecular Photonics
  • Quantum Information

In addition to its own resources, SDTech received labour from other Research Centres to supports its activities.

Between FY 2013-14 to 2016-17, over three-quarters of the labour supporting SDTech (78%) was internal (i.e., personnel belonging to SDTech ) and 22% was external to the Research Centre.

SDTech labour resources. Long description follows.
 
Long description of SDTech labour resources.

This figure is a horizontal stacked bar chart comparing the labour SDTech provides to different Research Centres between FY 2012-13 and 2016-17. This figure shows that SDTech provides a substantial portion of labour to support other Research Centres.

Research Centre Percentage Total amount in dollars
SDTech 78% $22.2 Million
AERO 7% $6.2 Million
AST 7%
ICT 6%
Other 2%

Source: NRC financial data system.

 

Over the period from FY 2013-14 to 201617, SDTech was ranked first among Research Centres in terms of the percentage of labour it received from other Research Centres and second overall (after the Energy, Mining and Environmental (EME) Research Centre) in terms of the percentage of labour it contributed to other Research Centres.

2.2.2 Facilities

SDTech facilities, located in Ottawa, are designed to help increase the understanding of light-matter interactions (used by the QPSS program) and to allow fabrication and prototyping for applications of advanced materials, including nanotechnology (used by the SMT program). Additional information on the facilities used by the QPSS and SMT programs can be found in Section 3.3.

2.2.3 Financial Resources

Between FY 2012-13 and 2016-17, SDTech generated approximately $11.37 Million in revenues, 70% of which was through the provision of strategic researchFootnote 13 services.

Type of service Fiscal year Total
2012-13 2013-14 2014-15 2015-16 2016-17
Technical Services $0.78M $0.58M $0.49M $0.84M $0.78M $3.47M
Strategic Research $1.85M $1.59M $1.39M $1.44M $1.63M 7.90M
Total $2.63M $2.17M $1.88M $2.28M $2.42M $11.37M
 

Source: NRC financial data system.

Expenditures associated with SDTech projects totalled approximately $50.6 million between FY 2012-13 and 2016-17. Nearly all these investments (88%) were allocated to strategic research projects.

3.0 Findings

The evaluation findings are organized around the following themes:

  • Need for R&D activities
  • Scientific excellence
  • Facilities excellence
  • Client and stakeholder engagement, and ecosystem development

Information is presented for SDTech as a whole as well as for the QPSS and SMT programs separately, where applicable. As SDTech provides a team of researchers to the Printable Electronics Program, information is also provided on its contributions to the achievement of outcomes.

3.1 Need for R&D activities

Given that Research Centres are aligned with the NRC 's strategic business goals, while programs are more transitory in nature and therefore more subject to change, the evaluation assessed the continued need for the R&D capabilities pursued by SDTech at the program level. The assessment was based on the results of the document review, interviews, a market assessment and the peer review committee's expert opinion.

3.1.1 QPSS Program

The QPSS program is addressing a need in the area of quantum photonics research and development, as quantum technologies are expected to have a significant impact on industries in the future.

The peer review committee (PRC) confirmed that, in quantum photonics, among the biggest technology development needs are in quantum sources and detectors, light-matter interfaces, and quantum memories, all of which are covered by the QPSS program. In addition, important non-linear optics applications are included within the scope of the program.

The PRC further stipulated that, in the future, various sectors of the Canadian economy will need quantum technologies to address their needs. Among the different technologies, quantum photonics offer promising approaches for addressing requirements of three important sectors: ICT , defence and security, and energy/environment. The QPSS program is considered to be well positioned to make significant impacts in these sectors as there is a critical mass of researchers and infrastructures at NRC to address the scientific challenges, and to build unique collaboration with Canadian companies and organizations.

While there are many social benefits associated with and potential commercial applications for quantum technologies, they are new, disruptive, and low- TRL in nature. As a result, R&D in the field carries a certain level of uncertainty and risk, making industry cautious about investingFootnote 14. In these situations the role of government becomes important for filling the gap in private sector investment by supporting basic research and conducting high-risk, low- TRL R&DFootnote 15.

A few internal and external interviewees felt that, while the research is promising, breakthroughs are not expected to occur for another several years, and perhaps not within the timeframe of the program. This was echoed by the PRC who felt that the business and scientific cases are clear, but are definitely long term in nature. As a result, they felt it might be too early to increase the TRL activity in order to achieve an impact on the Canadian economy. The exception is in the areas of resource, environmental and security sensing, where the program has developed technologies to a higher TRL level, which may result in impacts within the program's timeframe.

3.1.2 SMT Program

The SMT program is addressing stakeholder needs in the area of advanced materials research and development (R&D), but not to its full potential.

Addressing deficiencies and technical challenges associated with existing personnel and vehicle armour systems has been identified in government documents as a need by the Canadian militaryFootnote 16. While the literature acknowledges that existing armour systems are effective (i.e., in terms of stopping threats from projectiles the armour was designed to protect against), studiesFootnote 17 have underlined a number of challenges associated with existing personal and vehicle armour designs, particularly in the context of military applications. For example:

  • A soldier's personal protective equipment (PPE) (or body armour), which typically includes ceramic plate inserts, is heavy. This adds significantly to the total load carried and can restrict mobility. Studies have also shown that heavy loads carried or worn on the body can lead to more rapid fatigue and increased stress on the musculoskeletal system, resulting in higher risk of injury.
  • For vehicles, the additional weight of armour can compromise vehicle capabilities, including maneuverability, fuel consumption, transportability, and/or payload capacity.

The SMT program has focused on a specific technological need in the defence sector. Namely, it "was created to support the development of high-performance, cost-effective, and lightweight blast and ballistic protection systems, with a specific focus on advanced personnel protective equipment (PPE) and armoured vehicle (AV) products"Footnote 18.

The PRC found that the activities performed by the SMT program are fully complementary to ones performed at the DRDC (the program's primary client) and there is a clear synergy between their R&D activities. The PRC pointed out that the program has selected specific materials and applications within its business case and has focused on developments in those areas. The arguments made in the foundational program documentation continue to be valid.

In terms of implementation however, the PRC felt the program has not focused sufficiently on addressing the need for fundamental research to widen applications to other areas beyond the defense sector. The current volume of system-level work being conducted with DRDC is low and there is a weak connection between low level TRL research and higher level demonstration because of the lack of a scale-up facility (described in more detail in Section 3.3.2).

3.2 Scientific excellence

Scientific excellence of SDTech as a whole was measured through a bibliometric analysis of publications, awards and prizes, patents and licenses. At the program level, scientific excellence was assessed by the PRC through their review of background materials and participation in the site visit.

3.2.1 Scientific excellence – SDTech

Publications

SDTech researchers perform above the world average in terms of the relative impact of their published work.

SDTech as a whole scored above world levels in terms of the quality, scientific impact and excellence of its published works during the period from 2003 to 2014Footnote 19.

Dimension Indicator Score
Quality Average of Relative Impact Factors (ARIF) 1.77
Scientific Impact Average of Relative Citations (ARC) 1.7
Excellence Proportion within Top 10% of Highly Cited Publications (HCP) 2.12
 

Source: Science Metrix (2016) Bibliometric Analysis of NRC 's Performance in Emerging Technologies, 2003-2014.

Note: The ARIF (average of relative impact factors) is the average of the impact factors of the journals in which SDTech 's papers are published, compared with the average for all papers published in the same year in the same specialty. ARIF>1 means that SDTech papers are published in journals cited more frequently than the average journal.

The ARC (average of relative citations) is the average of the number of citations that each of SDTech 's papers received, relative to the average number of citations received by world papers published in the same year, in the same specialty. ARC>1 means SDTech papers are more cited than the world average.

The HCP score indicates that 21% of SDTech papers are included in the top 10% of most-cited publications worldwide.

 

At the research sub-field level, SDTech research generally has had a high impact, when compared with the world level during the 2003 to 2014 period.

Sub-field Number of Papers ARC
Materials 46 3.57
General Physics 239 3.02
General Science and Technology 38 2.73
Fluids & Plasmas 147 2.05
Optoelectronics and Photonics 42 1.91
General Chemistry 106 1.83
Optics 58 1.76
Polymers 30 1.49
Applied Physics 188 1.30
Organic Chemistry 145 1.11
Inorganic & Nuclear Chemistry 51 1.11
Physical Chemistry 127 1.09
Chemical Physics 288 1.02
Nanoscience & Nanotechnology 91 0.83

Source: Science Metrix (2016) Bibliometric Analysis of NRC 's Performance in Emerging Technologies, 2003-2014.

 
Awards and Prizes

SDTech researchers are recognized nationally and internationally for their research excellence.

SDTech senior researchers have garnered prestigious awards and prizes from organizations such as the Optical Society of America, the Russian Academy of Sciences, the Royal Society of London and the Royal Society of Chemistry. As information on prizes and awards is not systematically collected by the NRC , it was not possible to provide a quantitative assessment of achievements as part of the evaluation.

Most internal and all external interviewees reported that the quality of research produced by SDTech staff is excellent. They pointed out that SDTech hosts some of the world's leading researchers in the field, conducting innovative and leading-edge research with the support of superior NRC technicians to manage and maintain facilities and equipment.

While the Research Centre's standing in terms of academic excellence is currently high, the profile of the research staff in terms of age and years of service presents a potential risk in the mid- to longer-term due to the significant number of pending retirements. As of May 2017, nearly a third of SDTech R&D personnel (31%) would be eligible for retirement in March 2022 and nearly half (47%) will be eligible to retire by March 2027.

Patents and Licensing

SDTech has been successful in translating its research into promising technologies.

Between FY 2012-13 and 2016-17 SDTech secured a total of 38 patents for 18 different technologies

SDTech secured patents for each year.. Long description follows.
 
Long description of SDTech secured patents for each year.

This figure is a vertical bar chart comparing the number of patents issued each year between FY 2012-13 and 2016-17. This figure shows that SDTech secured patents for each year.

Year FY 2012-13 FY 2013-14 FY 2014-15 FY 2015-16 FY 2016-17
Patents 9 7 6 9 7

Note: Data shows a count of the total number of patents issued each year, which could include multiple patents for a single technology or invention.

Source: NRC -BMS IP data.

 

A total of 5 license agreements for technologies developed by SDTech researchers were secured between FY 2012-13 and 2017-18, including 2 associated with the materials thrust of the Printable Electronic Program

FY Technology
2012-13 Single Walled Carbon Nanotubes
2013-14 Purification of Single Walled Carbon Nanotubes
2015-16 Large Scale Synthesis of Boron Nitride Nanotubes
2015-16 Fiber Bragg Grating
2017-18 Conductive inks
 

Source: NRC Business Management Services IP data.

3.2.2 Scientific excellence – QPSS Program

The QPSS program is conducting world-leading research in the area of attosecond science. In addition, the research conducted in the fields of molecular photonics, quantum dot research and microscopy are considered outstanding.

The PRC noted that a number of QPSS researchers are recognized nationally and internationally for their excellence. In particular, both the PRC as well as a few internal and external interviewees mentioned Dr. Paul Corkum, who is considered the pioneer of attosecond science. PRC members noted that since he has introduced the 3 step model in 1993, Dr. Corkum has made several pioneering contributions to ultrafast molecular imaging and laser science. Recently, he has extended concepts from attosecond science in gas phase to the solid state. The PRC felt that this will provide novel metrologies for imaging and controlling materials relevant to ICT technologies.

The bibliometric study conducted in 2016 revealed that researchers in the areas of General Physics, and Fluids and Plasmas (most likely supporting QPSS ) have some of the highest impact scores of all SDTech publication areas, further reinforcing the PRC 's view.

In terms of recruitment and retention, the PRC found the QPSS program has been able to attract young promising researchers who have already contributed to the development of emerging quantum technologies including a random number generator. The committee felt the use of volunteer visitorsFootnote 20 has also allowed the program to continue to capitalize on the expertise of senior scientists. That said, an external interviewee noted that, while the NRC is currently able to attract researchers to the QPSS program because of the reputations of key scientists, this may not be the case once those individuals retire.

3.2.3 Scientific excellence – SMT Program

The SMT Program is conducting world-leading research in the development and application of boron nitride nanotubes (BNNTs). However, these efforts have not been as visible within the scientific community in the past few years.

World's first pilot scale production of BNNT

The nanocomposites team within the SMT program developed a breakthrough technology for the production of boron nitride nanotubes (BNNT) that was hundreds of times more efficient than previous methods. Canada is now the leading global producer of this game-changing advanced material, which combines ultra-high strength and unique electronic properties with versatile manufacturability. Industrial deployment will drive new, high-value manufacturing across the defence, aerospace, automotive and health sectors. The patented technology earned the team a Canadian Public Service Award of Excellence for scientific contribution.

Source: SDTech

The PRC found that the program has addressed relevant questions in the field of advanced materials and developed solutions to outstanding problems such as large-scale nanotube sorting, large-scale production of both CNT s and BNNT s. The papers reviewed by the PRC as part of the evaluation were found to be of high quality and the results robust and convincing. That said, some PRC members had the impression that the NRC scientists conducting research in the area of advanced materials are recently less visible at the international level and felt that additional scientific dissemination and engagement would support strengthening of low TRL activities. In addition, it was felt that the program risks depleting its core capabilities by too great a focus on higher TRL and technology transfer activities.

The PRC felt advances in BNNT s were well documented to support world leading status in research and application. The PRC noted the importance of the material transfer agreements that have been created for BNNT s which have had an impact on the research being conducted in this area. That said, the PRC also indicated that the number of patents may not be as high as expected for materials research and felt there may be impediments to filing new/provisional patents.

Program outputs including, material transfer agreements, patents and license agreements indicate that the level of work is above average.

Output Number
Material Transfer Agreements ( FY 2012-13 to FY 2016-17) 23
Maintained patents (as of April 2017) 14
License agreements ( FY 2012-13 to FY 2016-17) 2

3.3 Facilities excellence

The evaluation included an assessment of the facilities used by the QPSS and SMT programs, based on interviews with internal and external stakeholders, a review of program documentation, as well as the opinions of the PRC members, based on tours of key facilities, conducted during the site visit.

3.3.1 QPSS facilities

For the most part, the program's quantum photonics research installations are world-class. The lack of access to high performance computing equipment was identified as a weakness.

The QPSS program uses NRC R&D facilities located in Ottawa. The facilities are designed to help increase the understanding of light-matter interactions.

Name of facility
  • Attosecond Science Laboratory
  • Molecular Photonics Laboratory
  • Electro-Optics Materials Laboratory: Prototyping
  • Electro-Optics Materials Laboratory: Synthesis and Characterization
  • Quantum Information Laboratory: Cryorefrigeration
  • Quantum Information Laboratory: Optics
  • Fibre Photonics Laboratory

The PRC identified a variety of innovative approaches that have been used to create and maintain facilities. In particular, the committee identified the Joint Attosecond Science Laboratory ( JASLab ), the Molecular Science Lab, and the Quantum Security Technology Access Centre (QSTAC) as examples of innovative approaches. Each is briefly described below.

The Joint Attosecond Science Laboratory (JASLab) was created in 2008 through a partnership between the University of Ottawa and the National Research Council. Housed at the NRC 's Sussex Drive facility in Ottawa, it is the only lab in Canada to enable attosecond measurements, with a large variety of end-stations for testing various quantum sensing applications. The partnership between the two organizations was highlighted by the PRC as a novel approach to capitalize on funding opportunities that would otherwise have not been available to the NRC .

The Molecular Photonics Laboratory, also housed at the NRC 's Sussex Drive facility in Ottawa, was built over time through minor capital investments. The PRC highlighted this strategy as being effective in building the lab to the highest standards.

The Quantum Security Technology Access Centre, created in September 2016 at the NRC 's Montreal Road campus in Ottawa, is a joint effort between NRC , the Communications Security Establishment (CSE), and DRDC . The PRC highlighted the fact that QSTAC provides an experimental laboratory for technology testing and demonstration that NRC 's partners were not able to establish themselves.

In addition to the examples above, the PRC also highlighted the cross- NRC collaboration between SDTech and the ICT Footnote 21 Research Centre as an efficient way to secure access to epitaxy and nanofabrication facilities and maximize facility use.

Overall, the PRC found that the JASLab and the Molecular Photonics Laboratory are both world-class facilities providing unique equipment to perform forefront research in quantum photonics. They also identified the QSTAC facility as a promising collaborative effort.

While the PRC was positive in its assessment of most of the facilities, it also cautioned that the lack of access to high performance computing limits the program's ability to do modelling work, particularly within the quantum theory research group. This may have a negative impact on the achievement of results. This caution was echoed by some internal interviewees who specifically mentioned a high reliance on outdated high performance computing equipment, low bandwidth and unstable network connectivity. As well, SDTech operational plans describe issues related to the procurement of new equipment for high performance computing (HPC) capacity. More specifically, the documents indicate that SDTech 's Quantum Theory research group is highly reliant on outdated HPC equipment located at the NRC 's Sussex laboratories.

3.3.2 SMT facilities

The nanotechnology facilities were found to be comparable to those of other national institutions working in this field of research, with the exception of the production of BNNT s which was considered unique.

The SMT program uses NRC R&D facilities located in Ottawa. The facilities are designed to allow fabrication and prototyping for applications of advanced materials, including nanotechnology (e.g., protective coatings, and ballistic materials).

Name of facility
  • Nanotube Development Laboratory: Chemistry
  • Nanotube Development Laboratory: Characterization
  • Nanotube Production Laboratory
  • Electro-Optics Materials Laboratory: Carbon Nanotube CVD Synthesis

There was agreement among internal interviewees that while the labs and facilities themselves are not necessarily unique compared to other organizations (with a few exceptions), how the labs are customized and used is unique (e.g., the laboratory for producing BNNT s).

The PRC concurred that the NRC 's nanomaterials facilities are excellent but are equivalent to what is available in a few Canadian universities. That said, the PRC confirmed that the program's material development activities are at the state-of-the-art, as the program is unique in producing BNNT materials in a significant quantity for research purposes.

The absence of a scale-up facility is restricting the ability of the program to pursue technology transfer opportunities.

SMT program documents indicate that a scale-up chemical processing facility is critical to supporting program objectives, particularly under Thrust 1 (technology platform development). According to the SMT business plan, the scale-up facility will be used to produce required amounts of materials to support activities undertaken under program Thrusts 2-4 and to meet the needs of industry as part of technology transfer activities.

In 2015, SDTech submitted a proposal to NRC senior management for approval of $6.9 million in capital investments for the construction of a chemical processing scale-up facility as part of the Flexible Research Facility (FRF) project at NRC 's Montreal Rd. campus. While planning documents indicate that the facility is intended to support SMT , five other NRC Research Centres have expressed interest in participating in the project to support their own activities and programs, including Construction, Aquatic and Crop Resource Development, Energy, Mining and Environment, Aerospace, and Automotive and Surface Transportation. SMT quarterly reports indicate delays in the establishment of the facility, while SDTech 's operational plan for FY 2016-17 indicates that work with NRC Administrative Services and Property Management (ASPM) is ongoing to plan the FRF, including determining the location of the facility and specifications (e.g., designing three units to be used for the new nanomaterial production facility, determining equipment requirements).

Most internal interviewees from SDTech management and the SMT program echoed the concerns raised in the quarterly reports and identified the facility as critical to the success of the program. The PRC agreed that the lack of a scale-up facility for material synthesis and processing is restricting program momentum with respect to technology transfer, noting that without access to a scale-up facility, the expectation of the program should change. At the time of the evaluation the facility had not yet been approved.

3.4 Client and stakeholder engagement, and ecosystem development

A number of lines of evidence were used to assess client and stakeholder engagement and ecosystem development, including the document and data reviews, interviews and the PRC 's assessment.

3.4.1 SDTech

While, overall, clients and stakeholder value their relationships with SDTech and reported good alignment with their long term goals, immediate needs and expectations with respect to commercialization have not always been met.

PE Program - Silver molecular ink development with GGI International

NRC co-developed a molecular ink for screen printing, enabling high resolution prints for printed antennas, Radio Frequency (RF) circuits, in-mold electronics and frequency selective surfaces. NRC characterization of GGI prints enabled the development of several PE products. Collectively, 10 patents have been filed to date.

Source: SDTech

External interviewees felt the relationships developed with SDTech were positive and constructive and a few mentioned that these relationships have been on-going for a number of years. Many external interviewees felt there was a good alignment between their needs and the capabilities offered by SDTech , however a few acknowledged that understanding the needs of clients can be a challenge, particularly with respect to commercialization.

External interviewees identified the ability to move technologies to higher readiness levels as a barrier to success. Most external interviewees felt that SDTech 's key strengths are in conducting R&D activities and in increasing awareness and understanding of the technologies being pursued. A few acknowledged that commercialization may not be central to SDTech 's mandate and that private sector businesses may not be interested in engaging in low- TRL activities as the payoffs are not immediate.

While challenges were identified, SDTech has been successful in licensing technologies. The PRC pointed to the technological impact of the materials thrust of the PE program as an example of an outstanding achievement in this area.

The current policy on intellectual property and the matrix structure were identified as barriers to collaboration.

The PRC felt the current NRC IP policy may be a potential barrier to collaboration. In particular, they warned that the policy may not be sufficient to encourage scientists to fully exploit the potential of the technologies being developed by SDTech . Based on their past experiences in this area, PRC members mentioned spin-off policies, incentives to patent and employee benefits from IP revenue as important elements of IP policies.

External interviewees also identified the current IP policy as being a potential barrier to success. Some interviewees felt that NRC 's retention of IP made it difficult for clients to adopt and commercialize products.

The NRC 's Intellectual Property PolicyFootnote 22 outlines the principles and authorities regarding the creation, protection and management of intellectual property generated and acquired by the NRC . At the time of the evaluation, the NRC had committedFootnote 23 to collaborate with the Privy Council Office, Justice Canada and Innovation, Science and Economic Development to make appropriate changes to the IP provisions under the National Research Council Act.

In addition to the IP policy, internal interviewees also identified the matrix structure as a potential barrier to collaboration. While overall they felt the matrix structure had encouraged more collaboration, some internal interviewees found that negotiating for resources was difficult at times and that the process differed across Research Centres.

3.4.2 QPSS Program

The overall level of engagement with quantum stakeholders in Canada was found to be above average.

Client Engagement

The QPSS program's main sources of revenue ( FY 2012-13 to FY 2016-17) are Canadian federal government departments and academia for strategic research projects, and US government and academia for technical research projects.

Strategic Research

Strategic research project revenues. Long description follows.
Long description of strategic research project revenues.

This figure is a horizontal bar chart comparing the sources of Strategic Research project revenues by percentage between FY 2012-13 and 2016-17. This figure shows that the main sources of revenues for strategic research projects are the Canadian government and Canadian Academia.

Strategic Research Revenue Source Percentage
Government - Canada 63%
Academia - Canada 23%
Government - US 10%
Other 4%

Technical Services

Technical services project revenue. Long description follows.

Source: NRC financial data system.

Long description of technical services project revenue.

This figure is a horizontal bar chart comparing the sources of Technical Services project revenues by percentage between FY 2012-13 and 2016-17. This figure shows that main sources of revenues for technical services projects are US government and US academia.

Technical Services Revenue Source Percentage
Government - US 47%
Academia - US 18%
Industry - Canada 16%
Government - Canada 11%
Other 7%
 

Overall, QPSS generated a total of $4.96 million in revenues between FY 2012-13 and 2016-17: $3.15 million from strategic research projects and $1.81 million from technical service projects.

Given the low- TRL focus of the program, the PRC felt the level of engagement by government and academia was appropriate at this time in the program's lifecycle, but also noted that the level of private sector engagement is low. A few PRC members questioned whether the original goal of the program to move from TRL 2-3 to TRL 4-6, as outlined in the business plan, was realistic, given the overall timeframe of the program.

In terms of specific initiatives, the Quantum Security Technology Access Centre was identified by the PRC as a promising collaboration. Through QSTAC , partners are able to access critical technology foresight for their operational objectives. It also provides a mechanism to engage external academic collaborators. That said, the committee felt QSTAC lacked a clear strategy moving forward.

Ecosystem Development

The development of a national quantum strategy ("Quantum Canada") is the primary engagement activity being undertaken through the QPSS program. Quantum Canada is supported by the NRC , the Natural Sciences and Engineering Research Council (NSERC) and the Canadian Institute for Advanced Research (CIFAR). At the time of the evaluation, QPSS had undertaken two consultative activities, namely a survey of stakeholders, and a Quantum Canada symposium (150 participants) and workshop (55 participants). Additional workshops were in the planning phase.

The PRC felt that efforts made to engage stakeholders through the Quantum Canada initiative are of high importance, given the potential of quantum technologies. The NRC , and in particular the QPSS program, plays an important role in creating bridges between the different stakeholders. The PRC found the variety of stakeholders from universities, government labs and Canadian companies engaged to date was considered to be promising.

Concerns were raised regarding the level of engagement with entities that can help bridge the gap between scientific discovery and commercialization.

The PRC found that the program has clearly engaged in research towards low to near TRL development, as evidenced by the production of patents. Coming from a strong "academic type" of research to this level was felt to be an outstanding achievement. That said, while the program has been able to license some of its technologies (e.g., fiber Bragg grating and mid-IR lasers) to industry, the committee felt the overall the level of engagement and revenues received from the private sector in this specific area were low.

The PRC felt there may not be adequate specialized support provided to the program to engage with industry in order to move technologies closer to commercialization, particularly in the area of resource sensing. Further collaboration with the NRC 's engineering-focused research centres who work in the mid- TRL range (i.e., TRL 4 to 6) was identified as a potential way in which the program could improve its efforts in reaching out to industry.

3.4.3 SMT Program

While over time the SMT program was expected to increasingly engage industry, the program has not yet achieved this outcome and continues to rely on DRDC .

The SMT program was created as a joint initiative between the NRC and DRDC . The program business plan called for a focus on maturing core capabilities in nano-modified and hybrid engineered materials, processes, modelling and characterization during the early stages of the program. In parallel with internal and collaborative investments with OGD partners, early work with industry was to focus on a mix of cost-shared research at low/mid TRL and technical service delivery at high TRL to extend industry innovation/receptor capacity. Over the course of the program, emphasis was to shift toward industry technology transfer with the majority of activities later in the program directly engaging Canadian industry clientsFootnote 24.

Over the period from FY 2012-13 to 2016-17 DND and DRDC were critical for SMT program revenue generation.

Top revenue generating clients. Long description follows.
 
Long description of top revenue generating clients.

This figure is a horizontal bar chart comparing the top revenue generating clients for the SMT program, between FY 2012-13 and 2016-17. This figure shows that DND/DRDC are the source for most of the revenue (79%).

Client Percentage In Dollars
DND/DRDC 79% $4.97 million
Health Canada 8% $0.5 million
Other 14% $0.86 million

This continued high reliance on DRDC for revenues was raised as a concern by the PRC who felt that engagement with broader potential industries is not as strong as it should be, given the focus of the program. Only one major client, DRDC , for almost all of the SMT effort was cited as a concern, because of possible budget cuts. The PRC felt the distribution of revenues needs to be broadened to include a greater number of clients to ensure stability for the future.

The concern with budget cuts was well founded. Program documents revealed that budget cuts at DRDC had in fact had an impact on program revenues. Revenues received from DRDC for strategic research projects peaked in FY 2014-15, then fell considerably in the two subsequent fiscal years. The decline was somewhat mitigated through technical service projects.

While the program has been successful in developing the Security Materials Technology Roadmap (SMTRM), there is limited evidence of implementation beyond the sharing of information.

In 2015, the SMT Program, in collaboration with DRDC and other industry partners, led the development of the SMTRM Footnote 25 to "accelerate innovation, competitiveness, and productivity in Canada's security materials technology sector"Footnote 26. As part of the roadmap process, a two-day workshop was held with various stakeholders from Canadian and US industry, government and academia. In all, a total of 130 individuals participated in the workshop.

Based on the results of the workshop, the SMTRM identified future market needs and opportunities for PPE and vehicle armour. Additionally, specific roadmaps were developed for both PPE and AV , the intent of which is to inform Canadian industry of market needs to help them develop their R&D and commercialization strategies. The SMTRM also described the technical challenges associated with PPE and AV products and identified technology solutionsFootnote 27. These priority areas and potential technology solutions were meant to inform the implementation of the SMTRM .

External interviewees who were involved in the development of the SMTRM felt that the process helped government, academia and industry better understand everyone's needs and expectations moving forward. Internal interviewees found the SMTRM to be a useful tool to engage stakeholders and a "mechanism to build community and innovation".

To facilitate the implementation of the SMTRM , an online virtual collaborative environment was developed to promote collaboration among security materials stakeholders by allowing them to share information on technological developments, articles, reports, papers and opinions relevant to the security materials marketFootnote 28. The virtual environment was launched in November 2016.

One example of SMTRM implementation was provided during interviews with stakeholders. These interviewees revealed that information from the SMTRM was used to develop a call for proposals under the Build in Canada Innovation Program (BCIP)Footnote 29, managed by Public Services and Procurement Canada (PSPC). The call for proposals was successful in pre-qualifying a number of potential suppliers. As there is no mechanism in place to track the use of the SMTRM beyond NRC 's direct involvement, it was not possible to thoroughly assess its implementation. That said, based on their experience and expertise, the PRC felt that, while the development of the website for the SMTRM was a good first step, and installing a mechanism for capturing information on facilitated connections was encouraged, implementation efforts need to move beyond information sharing for the SMTRM to be successful.

4.0 Conclusions and Recommendations

The evaluation of SDTech revealed that, overall, the Research Centre is, through its two programs, addressing client needs and achieving scientific impacts. Efforts to engage key stakeholders are underway through the Quantum Canada and SMTRM initiatives. Technology development is in the early stages and presents some challenges moving forward, which is not surprising, given that the Research Centre is engaged in activities at low TRL levels.

With respect to each of the programs, the PRC provided the following conclusions and recommendations.

QPSS Program

Overall, the PRC concluded that the NRC should continue to support the foundational research conducted through the QPSS program, pursue efforts to engage partners and help coordinate and further develop a Canadian quantum ecosystem.

The PRC believed no significant changes were required to the program's overall strategy, however it felt there were opportunities for senior management to further maximize the program's impact.

With respect to fundamental research, the committee recommended that the NRC :

  1. Develop a strategy to ensure the world-leading status of the fundamental research carried out within the program is maintained. In developing the strategy, the NRC should:
    • Identify opportunities to engage with universities in the creation of collaborative arrangements similar to the Joint Attosecond Science Laboratory (JASLab) in order to further develop specific areas of research that are of interest to the NRC .
    • Review other successful collaborative research models such as JILA and the Joint Quantum Institute in the United States and identify elements that could be applied within the NRC context.
    • Explore methods to access required high performance computing, including, where possible, funding from external entities (e.g., Compute Canada).
    • Formulate a succession plan to address pending retirements of key researchers and ensure there is a pipeline of promising young researchers.

    While the PRC acknowledged that the program, for the most part, operates in a low-Technology Readiness Level (TRL) environment, it felt that engagement with clients and partners was an important element in ensuring the program remains relevant. Given the NRC 's mandate, in order to strike the right balance between academic interest and industry impact, the committee recommended that the NRC :

  1. Develop a strategy and execution plan for the Quantum Security Technology Access Centre (QSTAC) in order to maximize the impacts of the joint collaboration between the NRC and other government departments (OGDs), industry and academia.
  2. Develop a strategy to further advance technologies in the security, resource and environmental sensing areas in order to:
    • Collaborate more effectively with industry partners on specific projects
    • Engage with entities such as not-for-profit organizations and public-private partnerships that work in the mid- TRL space (i.e., TRL 4 to 6) to bridge the "valley of death" between scientific discovery and commercialization
    • Identify the type and level of specialized organizational support (e.g., Business Management Services) required to implement the strategy.

SMT Program

Overall the PRC concluded that the NRC should modify the SMT program.

The committee recommended the program's overall strategy and business plan be revised in order to:

  1. Place more emphasis on the fundamental research being conducted within the program as a foundational requirement for advanced materials development:
    • Consider innovative approaches to securing highly qualified personnel (e.g., adjunct professorships, increased use of visiting scientists)
    • Ensure access to high performance computing (and required theoretical expertise) to support the fundamental research
    • Explore opportunities for potential collaboration with the QPSS program (e.g., single photon sources based on hexagonal boron nitride)
  2. Refocus technology transfer efforts, given available resources and the program's lifecycle:
    • Determine whether the development of a scale-up facility required to develop and test prototype materials for clients is feasible within the timeframe of the program and if not, revise program objectives accordingly
    • Clearly articulate the program's role in technology transfer activities:
      • Consider the development of a more formal approach for engaging the engineering-based research centres within the NRC where appropriate
      • Consider innovative approaches for working with industry to further technology development and transfer (e.g., embedding industry experts in projects)
      • Consider the type and level of specialized organizational support required for technology transfer and business development (e.g., Business Management Services)
  3. Expand the program's client base beyond DRDC , particularly with the end-user community, where appropriate:
    • Capitalize on the development of the SMTRM to identify opportunities to diversify the program's industrial partners and enhance the value chain
    • Consider dual-use applications to ensure technologies developed have applications beyond the defense industry.

5.0 Management Response

QPSS Program

Recommendation Response and Planned Action(s) Proposed Person(s) Responsible Timelines Measure(s) of Achievement
Recommendation 1 - Recommendation accepted

Develop a strategy to ensure the world-leading status of the fundamental research carried out within the program is maintained. In developing the strategy, the NRC should:

A. Identify opportunities to engage with universities in the creation of collaborative arrangements similar to the Joint Attosecond Science Laboratory (JASLab) in order to further develop specific areas of research that are of interest to the NRC .

Develop a strategy, including:

A1. Exploring opportunities to establish an NRC Collaboration Centre with the Perimeter Institute in Waterloo.

A2. Retaining existing adjunct professorships and seeking to obtain additional adjunct professorships at Canadian universities.

Director General

A1. March 2019

A2. September 2018

A1. Discussions have taken place with potential university partners.

A2. Increase from 9 to 11 adjunct professors

B. Review other successful collaborative research models such as JILA and the Joint Quantum Institute in the United States and identify elements that could be applied within the NRC context.

Reviewing JILA and other models.

Program Lead

March 2019

Report completed and submitted to SDTech management team.

C. Explore methods to access required high performance computing, including, where possible, funding from external entities (e.g., Compute Canada).

Looking at potential avenues and selecting one or more of these avenues as appropriate.

Director of Operations

August 2019

  • High end PCs in place
  • Cluster available and meeting scientists' requirements.

D. Formulate a succession plan to address pending retirements of key researchers and ensure there is a pipeline of promising young researchers.

Formulating a succession plan to address pending retirements of key researchers.

Director General

March 2018
(Done yearly)

Succession plan included in the SDTech Quantum strategy.

Recommendation 2 - Recommendation accepted

Develop a strategy and execution plan for the Quantum Security Technology Access Centre (QSTAC) in order to maximize the impacts of the joint collaboration between the NRC and other government departments (OGDs), industry and academia.

Develop a strategy and execution plan for QSTAC as part of the proposal for the next iteration of QPSS (the Quantum Foundational Program).

Program Lead

September 2018

A QSTAC strategy is included in the Quantum Foundational Program proposal

Recommendation 3 - Recommendation accepted

Develop a strategy to further advance technologies in the security, resource and environmental sensing areas in order to:

  • Collaborate more effectively with industry partners on specific projects
  • Engage with entities such as not-for-profit organizations and public-private partnerships that work in the mid- TRL space (i.e., TRL 4 to 6) to bridge the "valley of death" between scientific discovery and commercialization
  • Identify the type and level of specialized organizational support (e.g., Business Management Services) required to implement the strategy.

The Quantum Foundational Program proposal will include a strategy for technology advancement.

Program Lead

September 2018

A strategy for technology advancement is included in the Quantum Foundational Program proposal.

 

SMT Program

Recommendation Response and Planned Action(s) Proposed Person(s) Responsible Timelines Measure(s) of Achievement
Recommendation 4 - Recommendation accepted

Place more emphasis on the fundamental research being conducted within the program as a foundational requirement for advanced materials development:

The next iteration of SMT will be the Adaptive and Intelligent Materials Foundational Program (AIM) which will have a greater focus on lower TRL work.

Program Lead

September 2018

AIM Program proposal emphasizes fundamental research

A. Consider innovative approaches to securing highly qualified personnel (e.g., adjunct professorships, increased use of visiting scientists)

A1. Retain existing adjunct professorships and seek to obtain additional adjunct professorships at Canadian universities.

A2. Have graduate students working with staff who are adjunct.

Group Leaders

April 2019

A1. 3 adjunct professors on staff in chemistry.

A2. A minimum of 6 graduate students working with adjunct staff.

B. Ensure access to high performance computing (and required theoretical expertise) to support the fundamental research

As indicated in Recommendation 1C: Looking at potential avenues and selecting one or more of these avenues as appropriate.

Director of Operations

August 2019

  • High end PCs in place
  • Cluster available and meeting scientists' requirements.

C. Explore opportunities for potential collaboration with the QPSS program (e.g., single photon sources based on hexagonal boron nitride)

Continue collaboration on BNNT for single photon sources. Opportunities will be explored as part of the yearly program technology project proposals, held in Q1 of each year.

Program Leads

June 2019

Opportunities for collaboration have been identified.

Recommendation 5 - Recommendation accepted

Refocus technology transfer efforts, given available resources and the program's lifecycle:

A. Determine whether the development of a scale-up facility required to develop and test prototype materials for clients is feasible within the timeframe of the program and if not, revise program objectives accordingly

Explore the possibility of co-funding the scale-up facility with DRDC and the instrument suppliers. If co-funding is not feasible, SMT will revise the program objectives and technology transfer strategy.

Director General and Program Lead

December 2018

Scale-up funding in place or new technology transfer strategy is in place as appropriate.

B. Clearly articulate the program's role in technology transfer activities:

B1. Consider the development of a more formal approach for engaging the engineering-based research centres within the NRC where appropriate

Discussions with Automotive and Surface Transportation (AST) and Aerospace (AERO) Research Centres to determine pertinent technologies to transfer.

Program Lead

September 2018

Contracts, project management data, correspondence with Engineering management.

B2. Consider innovative approaches for working with industry to further technology development and transfer (e.g., embedding industry experts in projects)

Continue to explore innovative approaches for working with industry to further technology development and transfer.

Program Lead in collaboration with BMS PBA

September 2018

Engagements will be documented as part of project documentation, contracts, NDAs and visitor agreements.

B3. Consider the type and level of specialized organizational support required for technology transfer and business development (e.g., Business Management Services)

Review the type and level of specialized organizational support required for technology transfer and business development and establish a renewed strategy.

Director General in collaboration with BMS PBA

April 2019

Support for technology transfer and business development strategy integrated as part of AIM Program proposal and SDTech Operational plan.

Recommendation 6 - Recommendation accepted

Expand the program's client base beyond DRDC , particularly with the end-user community, where appropriate:

A. Capitalize on the development of the SMTRM to identify opportunities to diversify the program's industrial partners and enhance the value chain

A1. Explore opportunities to expand business with existing customers (e.g., Dew, Nortrax and Mawashi) and continue its expanded business activities with Tekna.

A2. Develop additional engagements with SMTRM participants across the advanced materials value chain, using workshops and direct engagements with companies.

Program Lead in collaboration with BMS CRL

October 2018

Contracts with non- DRDC clients.

B. Consider dual-use applications to ensure technologies developed have applications beyond the defense industry

B. Explore alternate applications and assess how technologies developed can be used beyond the defense industry and approach new potential clients

Program Lead in collaboration with BMS CRL

October 2018

  • New applications for nanocomposites have been identified.
  • Technology projects have been launched where applicable.
 

Appendix A: Peer review committee Members and Site Visit Agenda

Peer Review Committee Members

Chair:

Denis Faubert
President, Chief Executive Officer
Consortium for Aerospace Research and Innovation in Canada

Members:

Tatjana Curcic
Director
Quantum Valley Ideas Lab

Jeff Fagan Project leader for the Particles, Tubes and Colloids project
Materials Science and Engineering Division
National Institute of Standards and Technology (NIST)

François Légaré
Professor
Institut national de recherche scientifique (INRS)

Richard Martel
Canada Research Chair on Nanostructures and Interfaces
Département de chimie Université de Montréal

Mark B. Ritter, Ph.D.
Distinguished Research Staff Member
Senior Manager, Physical Sciences
IBM TJ Watson Research Centre, Yale University

Christoph Simon
Professor
Institute for Quantum Science and Technology
University of Calgary

Site Visit Agenda

Tuesday, September 26, 2017

Time Event
19:00-21:00 Committee dinner and pre-meeting discussion
 

Wednesday, September 27, 2017

Time Event
8:15-8:30 Chair Welcome and review of agenda
8:30-9:00 Introductory remarks - Geneviève Tanguay, VP – Emerging Technologies
9:00-9:30 Presentation – Overview of SDTech - Duncan Stewart - General Manager, SDTech
9:30-10:15 Question period
10:15-11:15 In-camera session and coffee break
11:15-12:00 Presentation – QPSS Program Overview and Quantum Photonics research
12:00-12:30 Question period
12:30-13:30 In-camera session and lunch
13:30-14:15 Site Visit: Attosecond Lab, Molecular Lab (100 Sussex)
14:15-15:00 Presentation – SMT Program Overview and Advanced Materials research
15:00–15:30 Presentation – Nanomaterials research in support of the Printable Electronics program
15:30-16:00 Site Visit: Printable Electronics Lab (100 Sussex)
16:00-16:30 Question period and coffee break
16:30-17:15 In-camera session
19:00-21:00 Dinner
 

Thursday, September 28, 2017

Time Event
8:00–8:30 Recap of previous day
8:30-9:00 Site Visit: Nanotube facility (1200 Montreal Road)
9:00-9:30 Additional Q&A with SDTech General Manager, if required
9:30-12:00 Committee deliberations on findings and consensus building
12:00-13:00 Working lunch
13:00-14:00 Committee deliberations continue
14:00-14:30 Chair review of results of the committee's deliberations and discussion of steps to finalize the report
14:30 Committee adjourns
 

Appendix B: Methodology

Scope

The evaluation of the SDTech Research Centre and its two programs ( QPSS and SMT ) covered the period from fiscal year 2012-13 to 2016-17 inclusive. The evaluation was carried out in accordance with the NRC 's approved evaluation plan and TBS policies. The Research Centre and its programs had not been previously evaluated.

The evaluation questions were developed based on consultations undertaken during the planning phase of the evaluation and a review of key documents. The questions are listed in Table 1.

Table 1: Evaluation Questions

  1. To what extent is there a demonstrable need for the R&D capabilities offered by SDTech ?
  2. To what extent has SDTech developed a (recurring) process to capitalize on new and emerging areas of research, and develop programs that have the largest probable impact on Canada's future economy?
  3. To what extent is SDTech recognized for research excellence?
  4. Is NRC 's SDTech infrastructure (facilities and equipment) viewed as scientifically excellent, leading-edge and unique?
  5. Have SDTech and its hosted programs led to the transfer of knowledge, skills and technologies to its clients and partners (academia, government, and industry)?
  6. To what extent has the SDTech portfolio and its hosted programs led to: an increased awareness and understanding of SDTech , increased engagement of SDTech partners and clients, and greater alignment and coordination of SDTech communities?
  7. To what extent has the SDTech portfolio and its hosted programs developed technology platforms?
  8. Have the SDTech portfolio and its hosted programs produced outputs that are realistic and appropriate given the level of resources?

Methodology

Data collection for the evaluation was conducted by an independent evaluation team from NRC 's Office of Audit and Evaluation, supported by an external consultant. The evaluation employed both qualitative and quantitative research methods as follows:

  • Document review (including program and portfolio plans and strategies, publicly available industry and government reports)
  • Analyses of financial, administrative and performance data (portfolio, program and project level financial and administrative data including revenues, expenses, number and type of projects, results of a bibliometric study, HR data)
  • Interviews with SDTech senior management and staff and external clients and partners, selected in consultation with senior management and a review of project data (internal staff/management n=19, external partners/stakeholders n=15)
  • Market assessment (a review of market assumptions made at the outset of the programs)

Data collected by the evaluation team was analyzed and summarized in a series of technical reports. These technical reports, along with other background information, was provided to a peer review committee comprised of seven individuals who possess expertise in SDTech 's research areas and/or the management of scientific research programs. A representative from the NRC Office of Audit and Evaluation (OAE) acted as the Peer Review Coordinator.

The Committee's membership was drawn from national and international academia, other government laboratories/departments, and relevant public and private sector organizations. Committee members were expected to participate in the review process in an objective, unbiased and credible manner, with no apparent or perceived conflict of interest. To this end, all members signed a non-disclosure/non-conflict of interest agreement.

Efforts were made to form a balanced committee, taking into consideration sector (academia, government, industry), research areas, and geography.

Peer Review Committee Member Tasks

The peer review process included three components: the review of the background material collected by the evaluation team, participation in a site visit to the NRC , and the production of a peer review report. The total level of effort required by committee members, including the site visit, was approximately 5 days. As part of the Peer Review Committee, members were responsible for:

  • reviewing briefing materials and becoming familiar with the research activities and management practices of SDTech and its programs
  • participating in conference calls in advance of the site visit
  • attending and actively participating in the peer review process, including the site visit (which occurred from September 26-28, 2017)
  • providing input into the peer review report
  • reviewing the draft peer review report and providing written comments

Review of Background Materials

Background materials were sent out to peer review committee members for their review prior to the site visit. Members reviewed the materials and completed and "Initial feedback form for Peer Review Committee Members". The purpose of the feedback form was to identify gaps in information that can be filled either through the provision of additional background documents, or through the site visit. The gaps were discussed by the committee during a teleconference call.

Initial Assessment

Peer review committee members then asked to conduct a preliminary assessment of SDTech and its programs, based on the background materials, using an assessment grid created specifically for the evaluation. The assessment grid contained a series of questions and required members to indicate an overall score (using a 3 point scale – below average/average/above average), justification for the score and areas to be further explored during the site visit. A teleconference was held to discuss the assessments and identify key areas to pursue in more detail during the site visit.

Site Visit

The peer review process included a 1.5 day on-site visit to the NRC in Ottawa, Ontario. The visit, which was preceded by a welcome dinner on September 26, occurred on September 27 and 28 and included presentations and discussions on past, current and proposed research activities, and tours of SDTech facilities.

Finalizing the Committee Report

The draft report was formatted and edited following the site visit and additional background information on the purpose and methodology was added to create a complete draft. A report was produced for each of the two programs in addition to an overall evaluation report for SDTech as a whole.

Study Limitations

The following limitations were identified during the course of the project. Where possible, mitigation strategies were used to minimize their impact, as described below.

  • Some of the information provided, particularly with respect to bibliometric data, was somewhat outdated. While a more recent bibliometric study was originally expected to be available in time for the review, questions were raised regarding the methodology used, precluding it from being included in the peer review process. In order to minimize the impact, additional information on key researcher accomplishments (awards, prizes) was provided to the peer review committee members.
  • Comparisons of the program's performance with that of other similar organizations was not conducted as part of the data collection phase of the project. The peer review committee members' knowledge of other organizations was relied upon to make any necessary comparisons.
  • The use of peer review committees can introduce unconscious biases into evaluation studies. In order to mitigate this potential result, efforts were made to ensure members were drawn from a variety of organizations and geographic regions. Significant effort was also made to ensure female representation on the committee.
  • Only a limited number of external interviews could be conducted during the timeframe of the project. Efforts were made to triangulate information across several lines of evidence in order to provide summary information to the peer review committee.
  • The length of the site visit was limited to 1.5 days (with a working dinner the night before) in order to minimize the burden on committee members and work within budget limitations. In order to minimize the impact, committee members were provided with materials prior to the site visit and responses to questions arising from their initial review of the materials were provided in writing prior to the site visit.

Appendix C: SDTech Logic model

""
Long description of the SDTech Logic model

The diagram shown in Appendix C is a logic model for SDtech. The logic model shows the progression from portfolio inputs, through activities to outputs and finally to three progressive levels of expected outcomes (immediate, intermediate, and ultimate).

The Logic Model is divided into 6 progressive levels which are listed below. Each level feeds into the following level.

  • Level 1: Inputs
  • Level 2: Activities
  • Level 3: Outputs
  • Level 4: Immediate outcomes
  • Level 5: Intermediate outcomes
  • Level 6: Ultimate outcomes

Level 1: Inputs

Inputs are demonstrated through the following indicators:

  • 1.1 NRC Investment (program, capital & net loss)
  • 1.2 Earned revenues (FFS technical services, cost-shared strategic R&D projects & other revenues)
  • 1.3 Common services & external support ( NRC & SSC)
  • 1.4 Contributions from other NRC portfolios & collaborators/partners (in-kind & cash contributions)
  • 1.5 People (HQP, specialized expertise & experience)
  • 1.6 Facilities (specialized facilities & equipment

Level 2: Activities:

The second level of the diagram is a list of activities, including:

  • 2.1 Human resource management (expertise/workforce development, staffing & strategic succession planning) – Portfolio level activity
  • 2.2 Facility management (development & management, health and safety) – Portfolio level activity
  • 2.3 Strategic planning (foresight, market intelligence) – Portfolio level activity
  • 2.4 Program & portfolio management/coordination & project management – Portfolio level activity
  • 2.5 Create knowledge (strategic R&TD projects: Internal/foundational research platforms/projects & collaborative strategic projects, technical services) – Program level activity
  • 2.6 Transfer knowledge – Program level activity
  • 2.7 Catalyse knowledge ecosystems – Program level activity

Level 3: Outputs

The third level of the diagram is a list of outputs:

  • 3.1 Enhanced/maintained competencies (utilized, trained & engaged HQPs) – Portfolio level ouput
  • 3.2 Enhanced/maintained world-class R&TD infrastructure (accessed/utilized facilities/equipment) – Portfolio level output
  • 3.3 Portfolio & Project management reports/metrics – Portfolio level output
  • 3.4 SDTech strategic plans/technology roadmaps – Program level output
  • 3.5 Strategic relationships/partnerships (with academia government & industry) – Program level output
  • 3.6 Identified & reached SDTech key players – Program level output
  • 3.7 Scientific & technical knowledge (publications, reports & advices/guidance) – Program level output
  • 3.8 Technology prototypes (new or improved technology/solutions) – Program level output
  • 3.9 Intellectual property (patents, trade secrets) – Program level output

Level 4: Immediate outcomes

The fourth level of the diagram describes immediate outcomes which are intended to take place between 1 to 5 years.

Immediate outcomes include:

  • 4.1 Improved execution of R&TD projects
  • 4.2 Research & technical excellence
  • 4.3 Technology platforms
  • 4.4 NRC is recognized as a national leader & hub for SDTech research
  • 4.5 Greater awareness & understanding of SDTech (Quantum & SMT knowledge base)
  • 4.6 Increased engagement of clients/partners (R&TD program development & implementation)
  • 4.7 Greater coordination & alignment of SDTech communities (quantum cluster, security material community)
  • 4.8 Transferred knowledge & technical skills for capacity development (to academia, government and industry)
  • 4.9 Transferred technology (to government and industry)

Level 5: Intermediate outcomes

The fifth level of the diagram describes intermediate outcomes which are intended to take place between 6 to 10 years.

Intermediate outcomes include:

  • 5.1 NRC is recognized internationally for its world-class R&TD in SDTech
  • 5.2 Contributing to solving Canadian public problems (health, environment and security monitoring, public safety, national security)
  • 5.3 Improved innovation capabilities for Canadian firms, Canadian Government agencies & academia
  • 5.4 Increased productivity of Canadian companies
  • 5.5 Creation & growth of Canadian firms (increased revenues & jobs creation)

Level 6: Ultimate outcomes

The sixth and final level describes ultimate outcomes which are intended to take place between 10+ years.

Ultimate outcomes include:

  • 6.1 Improved economic, social & environmental prosperity for Canada
  • 6.2 Sustainable SDTech technology innovation capacity in Canada