Evaluation of the NRC’s Medical Devices Research Centre

 

Alternative format

PDF version (PDF, 1.16 MB)

 

Prepared by: Office of Audit and Evaluation, National Research Council Canada

Approval: This report was approved by the NRC's President on September 22, 2019

© 2019 Her Majesty the Queen in Right of Canada, as represented by the National Research Council of Canada.

Cat. No. NR16-298/1-2019E-PDF

ISBN 978-0-660-33076-1

Acronyms

AST Automotive and Surface Transportation Research Centre
FTE Full-time equivalent
FWCI Field-Weighted Citation Index
GBA+ Gender-based analysis plus
GRDI Genomics Research and Development Initiative
HQP Highly qualified personnel
MD Medical Devices Research Centre
MNE Multinational enterprise
NRC National Research Council of Canada
OAE Office of Audit and Evaluation
OGD Other government department (Canadian federal)
PRC Peer Review Committee
SME Small-to-medium enterprise
TRL Technology Readiness Level

Executive Summary

MD supports Canadian medical device companies, other government departments, and multinational enterprises to develop innovative medical technologies that provide rapid, sensitive, accurate and low-cost solutions aimed at saving lives, reducing healthcare burden and stimulating economic opportunities for Canada. Its research is organized into three thrusts: In Vitro Diagnostics, Implantable Devices, and Simulation and Digital Health. It is the NRC's smallest research centre in terms of both budget and staff.

This evaluation covered 2012-13 to 2017-18, inclusively, and drew on a bibliometric study, case studies of projects with 6 clients, data review, document/literature review, internal and external interviews, and a peer review by experts from industry, government, and academia.

Areas of demonstrated strength

Performance
MD's research was leading edge and contributed to advancements in the medical device field. It also had a positive impact on clients, supporting business innovation and growth of the medical device industry. MD's research is positioned to support government policy solutions in the future.

Uniqueness
There is a need for MD's research, which has unique characteristics for example that differentiate it from other players in the field. MD can play an even greater role in the medical device industry if it were to grow.

Areas for improvement

Appropriateness of research
MD's research made significant contributions to the medical device field. MD can refine the focus of its research within each of its 3 thrusts to have a greater impact.

Recommendation 1: Devise and implement a strategic planning process per research thrust, and update these plans annually. Plans should identify opportunities to exploit (e.g., In Vitro Diagnostics' lab-on-a-chip technology), reconsider and revise research focus where appropriate (e.g., Simulation and Digital Health, and Implantable Devices), and consider involvement of and implications for end users (e.g., patients, marginalized populations).

Engagement
MD has worked with appropriate stakeholders to date. MD can increase its outreach and communication efforts for greater visibility as well as work more with pharmaceutical companies, major diagnostic firms and community health organizations.

Recommendation 2: Develop stakeholder engagement plans, per research thrust, and report against progress made to ensure continued alignment with the strategic plan.

Recommendation 3: Identify strategies to increase awareness of its capabilities within relevant industries, in particular those where its profile is low but where there are opportunities for growth.

Capabilities
MD had the appropriate capacities, competencies and facilities to meet its objectives. There are opportunities for MD to grow in size to have an even greater impact. Heavy dependence on key staff poses a risk to its ability to succeed, should they leave. MD must invest in its current facilities to remain state-of-the-art, which is necessary to continue its research in the future.

Recommendation 4: Develop a strategic plan for staff development and succession planning for each of its thrusts.

Recommendation 5: Prioritize investments in its major facility for In Vitro Diagnostics, the BioAnalytical Clean Room, to support current work and to allow for future growth.

Introduction

An evaluation of MD was conducted in 2018-19. It assessed the relevance and performance of the research centre. This reports provides an overview of the main findings and conclusions as well as recommendations for MD.

An evaluation of the MD Research Centre and its Health Technologies program was conducted in 2018-19. The evaluation period covered 2012-13 to 2017-18, inclusively. The evaluation was carried out in accordance with the NRC's approved evaluation plan and Treasury Board policies. MD had not been previously evaluated.

This report begins by providing a profile of MD. It then presents the evaluation findings on MD's uniqueness, appropriateness of research, stakeholder engagement, capabilities, and performance. Following the conclusion are five recommendations for improvements within MD.

Evaluation approach

Methods
In order to maximize the possibility of generating useful, valid and relevant evaluation findings, mixed methods were used. This allowed for convergence of results across lines of evidence and developing a better understanding by exploring different facets of complex issues.

  • Bibliometric study (publication citation analyses)
  • Case studies (on projects with six clients)
  • Data review (administrative and performance data)
  • Document/literature review
  • Internal and external interviews
  • Peer review

For more detailed information on the methods, including challenges and limitations, refer to Appendix A.

Questions
The evaluation questions were developed based on consultations and a review of key documents. The questions were:

  1. To what extent has MD scoped its research in the most appropriate areas?
    1. How does the work conducted by MD/HT differ from that conducted by others (e.g., universities)?
    2. Are MD/HT focused on the 'right' areas?
  2. Has MD/HT engaged with the right clients, collaborators, and other stakeholders?
  3. To what extent does the research centre have the capacities, competencies, and facilities to achieve its objectives?
  4. To what extent is MD a leader in scientific excellence in the fields of In vitro diagnostics, implantable devices, and simulation and digital health?
  5. Has MD/HT contributed to (or is it positioned to contribute to) the economic growth and prosperity of the Canadian health technologies/medical devices industry?
  6. Has MD/HT contributed to (or is it positioned to contribute to) government policy solutions?

Profile

MD supports Canadian medical device companies in their quest for new sources of productivity, competitive advantage and growth by providing customized research and technology solutions. They help their clients develop innovative medical technologies that provide rapid, sensitive, accurate and low-cost solutions aimed at saving lives, reducing the healthcare burden and stimulating economic opportunities for Canada.

Activities and resources

Research activity
MD's research activity has been organized into the Health Technologies program. It has 3 research thrusts:

  • In vitro Diagnostics: Develop scalable and affordable bioanalytical solutions that may be deployed in settings such as hospital point-of-care, clinical diagnostics, drug discovery and development, and life sciences applications.
  • Implantable Devices: Develop, manufacture, and test biocompatible materials for orthopedic devices.
  • Simulation and Digital Health: Offer product development services that address emerging business opportunities in surgical efficiency techniques, medical technology software, health IT and homecare rehabilitation.
  • A spectroscopy thrust (In Vivo Imaging) was maintained between 2012-13 and 2016-17 before that area of research was discontinued.

Financial resources
Between 2012-13 and 2017-18, MD had expenditures of $81.0 million and generated $16.7 million in revenues. Revenues were generated from a mix of clients from industry (46%), academia and others (32%), and OGDs (22%). MD is the NRC's smallest research centre in terms of both budget and staff. MD represents 1% of the NRC's total annual budget and 2% of the NRC's total staff.

Revenues and Expenditures ($M) 2012-13 2013-14 2014-15 2015-16 2016-17 2017-18 Total
Revenues $2.8M $3.3M $1.8M $2.7M $2.6M $3.5M $16.7M
Industry 34% 20% 21% 38% 71% 81%* 46%
Other government departments 36% 23% 25% 17% 20% 12% 22%
Academia and others 30% 57% 50% 45% 9% 7% 32%
Other sources 0% < 1% 4% < 1% < 1% < 1% < 1%
Expenditures $21.8M $16.0M $12.3M $10.4M $9.6M $10.9M $81.0M

*Note: FY2017-18 industry figures includes $730 thousand from IRAP's R&D Certificate Program to offset discount to Canadian SMEs.

The higher costs in 2012-13 to 2014-15 were attributable to:

  • satellite locations in Calgary, Halifax, and London, and larger presences in Winnipeg and Ottawa, that were associated with the former institutes that formed MD (and which were subsequently disbanded)
  • maintenance of equipment later deemed outside scope of MD (e.g., MRIs under the In Vivo Imaging thrust)

Human resources and facilities

Human resources
MD has a total of 69 staff (as of March 31, 2018) in three locations. It has two sections, BioAnalytical Micro Nano Devices and Simulation and Digital Health, that deliver two of the Health Technologies program's research thrusts – In Vitro Diagnostics and Simulation and Digital Health. The Health Technologies program draws on MD resources for the most part (86% of its labour between 2012-13 and 2017-18 comes from MD). Select staff from the NRC's AST research centre are primarily responsible for implementing the Health Technologies program's third research thrust, Implantable Devices, with support from MD staff. See Appendix C for MD's organizational chart.

Location map
Long description of Location map

Boucherville , QC

  • 70% of staff
  • Facilities and equipment include:
    • bioanalytical micro-devices clean room
    • polymer-based micro and nanofabrication facility
    • polymer lab-on-a-chip and microanalytics laboratory
    • microfluidic prototyping facility
    • molecular diagnostics (clinical, food, veterinary, environmental) laboratory
    • object-oriented software platform for interactive simulation
    • simulation and interactive haptics laboratory
    • soft tissue biomechanics laboratory
    • tissue-mimicking phantoms facility
    • connected health and cognitive health
    • implantable biomaterials and manufacturing processes laboratory
    • development and characterization facilities for metals, polymers, ceramics and their composites

Winnipeg, MB

  • 27% of staff
  • Facilities and equipment include:
    • computer-aided engineering, design and simulation software
    • spectroscopic technologies (Visible, IR, Raman, CARS, OCT)

Ottawa, ON

  • 3% of staff
  • No facilities and equipment

Projects and clients/collaborators

MD has completed or implemented 147 projects with 112 unique clients and collaborators from 2012-13 to 2017-18. It has shifted over time from working primarily with academia to industry. Overall, most projects were strategic R&D (63%) though MD also provided technical services, mainly to its industry clients (37%).

Industry including 51 SMEs and 16 MNEs

  • 96 projects: 58% R&D, 42% technical services
  • $7.7 million in revenues (average revenue/project: $71K)

Academia and others including 16 universities, 13 hospitals and research institutes, and 11 other organizations.

  • 39 projects: 74% R&D, 26% technical services.
  • $5.3 million in revenues (average revenue/project: $126K)

Other government departments including 4 federal clients (i.e., Canadian Food Inspection Agency, Canadian Space Agency, Defense Research Development, Health Canada)

  • 12 projects: 92% R&D, 8% technical services
  • $3.6 million in revenues (average revenue/project: $261K)

What is strategic R&D vs technical services?

Strategic R&D consists of collaborative research projects undertaken with partners to de-risk R&D and accelerate commercial development timelines. Technical services consist of projects that assist clients in solving immediate technical problems through the delivery of specialized fee-for-service support (e.g., testing and certifications, calibration, prototyping, demonstrations, scale-up and consulting).

Uniqueness

Overall finding: There is a need for MD's research, which has unique characteristics that differentiate it from other players in the medical device field.

Sources: Internal and external interviews, data review, document review, peer review.

MD has unique characteristics that differentiate it from other organizations conducting research in similar areas.

Availability of facilities/equipment
MD's facilities and equipment are maintained by technical officers, as opposed to students, as is the case at universities. This allows for continuity of operations and technical expertise. The presence of a fabrication team also allows MD to customize and build necessary pieces from scratch to provide more complete services than others.

Breadth of expertise
MD has a wide range of multidisciplinary and long standing expertise. As a result, it can work with stakeholders, ranging from academics, to clinicians, to industry.

Absorbs risk
MD has the ability to do early stage research and absorb the risk that industry is unable to or unwilling to.

Technology readiness level
MD generally works within the TRL level 4-6, bridging the gap between early research (1-3) and later development (7-9). MD's work with TRL 4-6 is aligned with the needs of industry. The number of strategic R&D projects MD had increased from three to 23 between 2012-13 and 2017-18, underscoring industry's interest in innovative R&D. The research centre's planned move to focus on lower TRL (1-3) under its most recent strategy (2019 to 2024) is necessary to develop new technologies to stay relevant and provide innovative solutions to industry, which is populated with companies driven by product innovation.

Scale and capacity
MD can go from concept to prototype and therefore work with clients on projects that span the TRL scale.

"MD is professional and very responsive. They are scientifically robust and they understand […] business needs…."—MD Client

MD's collaboration centres will reduce the potential for duplication of effort between the NRC and other organizations conducting research in similar areas.

MD is one of few Canadian organizations conducting research in the areas of Implantable Devices, and Simulation and Digital Health. There are, however, other organizations operating in the area of In Vitro Diagnostics, like MD. Rather than compete, MD is partnering with leading institutions in the field including the Centre Hospitalier Universitaire Sainte-Justine (announced July 2017) and the University of Toronto, to form the Centre for Research and Applications in Fluidics Technology (CRAFT; announced November 2018).

These partnerships will allow MD to:

  • leverage complementary know-how, tools and expertise
  • strengthen ties with academia
  • accelerate the innovation process for the development and commercialization of new technologies and applications in microfluidics
  • promote the recruitment and training of post-doctoral fellows and graduate students
  • create an innovation ecosystem

"In the microfluidics section, some of the work done is quite unique, very early stage and of significant interest" —External Interviewee

Appropriateness of research

Overall finding: MD's research made significant contributions to the medical device field. MD can refine the focus of its research within each of its three thrusts to have an even greater impact and address opportunities in the medical device field. A systematic strategic planning process will facilitate this refinement.

Sources: Document/literature review, internal and external interviews, peer review.

Relevance

Overall, MD's research addressed societal needs as well as areas of importance for the medical device industry. Within each of MD's research thrusts, there are opportunities for further refinement for even greater impact. MD's potential is far greater than what is currently realized, and would require MD to grow.

Disbanded research thrusts
MD's decision to disband its In Vivo Imaging thrust in 2016-17 was appropriate because the domain is saturated with other players.

MD's decision to not pursue a testing and certification thrust was based on two factors: a potential overlap with other organizations in industry, and a lack of relevant competencies within the NRC. There may be an opportunity for the Implantable Devices thrust to collaborate with others (e.g., Health Canada) in this domain.

Current research thrusts
MD's current research is aligned with challenges and opportunities in the medical device sector, including a change in healthcare approach, high import dependence, a need to commercialize research and an aging population.

All 3 of MD's research thrusts have made significant contributions to their field. Their respective focuses are well aligned and based on existing capabilities and strengths of the NRC (i.e., those from previous institutes).

There are opportunities for each of MD's research thrusts to refine its focus in support of even greater impacts. This would also require MD to grow. These include:

  • Simulation and Digital Health thrust – focus on digital therapeutics (of which virtual reality is only a small part of), including contactless and visual monitoring of bodily signals, vital signs, and data stream analysis.
  • Implantable Devices thrust – have a broader focus that includes support for medical devices in the area of material processing and materials testing for implantable devices.
  • In Vitro Diagnostics thrust – further exploit microfluidics and lab-on-a-chip technology.

Further details on the relevance of each thrust are provided on the subsequent pages.

Relevance – Simulation and Digital Health research thrust

Current research
MD's work on interactive simulation addresses a broad range of areas that have importance for industry, as well as the Canadian society and economy. These include:

  • medical task simulation (e.g., use of virtual reality for surgical training)
  • cognitive care (e.g., use of virtual reality for psychological assessment)
  • interactive remote care (e.g., management of remote patients)

Opportunities
The PRC concluded that MD would benefit from re-examining its research strategy and refocusing its research in simulation and digital health in specific areas that try to solve grand challenges. This would enable it to have a greater impact as opposed to the incremental contributions it has had to date. It would also contribute to MD finding the next breakthrough. MD could refocus its efforts in:

  • digital therapeutics (of which virtual reality is only a small part of), including contactless and visual monitoring of bodily signals, vital signs, and data stream analysis
  • assessments/monitoring of psychiatric and pharmaceutical therapies, which have a bigger market than psychological therapies (as currently targeted by the thrust)

Relevance – Implantable Devices research thrust

Current research
MD's work in implantable devices is aligned with some research areas that have current/future market growth potential and address societal needs. This includes:

  • orthopedics (e.g., titanium foam)
  • dental implants (e.g., porous coating)

Opportunities
The PRC found that MD needs to think more strategically about the role the Implantable Devices thrust can play in the medical device field, and give it a broader focus than it currently has. This could include:

  • support for medical devices in the area of material processing
  • materials testing for implantable devices (e.g., in collaboration with Health Canada as part of implantable device certification)

Expansion of the Implantable Devices thrust's mandate would require additional resources. It would, however, allow the Implantable Devices thrust to have a concentrated impact, as opposed to incremental contributions.

Relevance – In Vitro Diagnostics research thrust

Current research
MD's research in microfluidics and lab-on-a-chip are focussed on areas that are important for industry stakeholders, as well as the Canadian economy and society. These include:

  • genetic testing (e.g., sample preparation to break the cell and extract nucleic acids where DNA and genes are found)
  • infectious disease (e.g., tests for latent tuberculosis and for sepsis)
  • oncology (e.g., creation of more sensitive tests in clinical oncology settings)

Opportunities
The PRC concluded that MD's work in the In Vitro Diagnostics thrust has the potential to seed a multi-billion dollar industry for Canada. MD can further exploit its microfluidics and lab-on-a-chip technology. The PRC specifically commented "MD's potential is far greater than what is currently realized. They have microfluidic technology that has applications for MNEs beyond their current SME base. Their expertise is advanced enough and technology sufficiently mature to attract MNE investment and collaboration".

The work of the In Vitro Diagnostics thrust is earlier in its development than the work of the Simulation and Digital Health thrust, so there is naturally greater future potential.

Strategic planning

MD should use a more structured approach and process to identify and prioritize industry needs to ensure continued relevance.

Identification of research priorities
Over the past 5 years, MD has evolved from technology push to using a combination of technology push and market pull when defining its research focus. MD has identified the needs of industry through ongoing interactions with stakeholders and involvement at trade shows and conferences.

The PRC concluded that MD would benefit from re-examining its research strategy, finding that only one of MD's 3 thrusts (In Vitro Diagnostics) had a long-term strategic vision. The In Vitro Diagnostics thrust is focused on addressing large scale problems (e.g., diagnoses of infectious diseases, food safety). The focus of the other two thrusts were more opportunistic in their research priorities, focused on incremental contributions to the field as opposed to addressing big problems. For the Simulation and Digital Health thrust, this was a strategy to validate the next area for which they should focus their efforts on. This also provided a source of revenue while MD worked on other earlier stage research such as its microfluidics and lab-on-a-chip technologies.

Strategic planning process
Across all thrusts, there is limited evidence of a structured and systematic strategic planning process. There would be value in each thrust having its own annual and long-term strategic plan.

Stakeholder engagement plans, intended to structure outreach efforts and identify industry needs, have not been maintained since the HT program's launch in 2015-16.

While there is no evidence that MD missed relevant opportunities as a result of this, it presents a risk for the research centre—that MD will not be aligned with the needs of the medical device industry, affecting its ability to achieve its objectives.

Going forward, the newly created MD Advisory Board (2018-19) can be used to support MD's efforts to identify and prioritize industry needs, and thus inform a new stakeholder engagement plan.

What is technology push vs market pull?

Technology push is when inventions are pushed through R&D. Market pull is when new inventions are developed in response to an identified need.

Consideration of end users

When identifying its strategic priorities, MD would benefit from consulting with and giving greater consideration to end users of its technologies. This includes those from diverse (GBA+) populations, consistent with the federal government's priority to consider GBA+ in all of its activities.

There are opportunities for MD to consult with end users and focus/tailor its research and pursuit of technologies accordingly.

There is limited evidence that MD consulted with end users in the design and implementation of its objectives and research. Often MD considered the perspectives of clinicians, viewing them as the end-users of their technology; they did not however consider patients as end users. It is important for MD to consider both populations when planning its research projects.

In addition, the technologies that MD is working on have particular implications for specific groups of people, of which are often diverse populations. It is a federal government priority to consider diverse groups (i.e., GBA+) in all of its activities.

While not a required consideration at the time of MD's inception, there are opportunities for MD to systematically identify and prioritize the needs of specific stakeholders within GBA+, and focus/tailor its pursuit of technologies accordingly.

For example:

  • Indigenous communities - MD's work on latent tuberculosis has the potential to address an important need within the indigenous community (tuberculosis has a disproportionate burden in indigenous communities, compared to general population). Likewise, MD's work on Point of Care solutions and interactive remote care has high implications for indigenous populations and/or remote areas.
  • People with disabilities - MD's work on orthese hallux valgus (forefoot deformity orthopedics) has implications for people with disabilities.
  • Elderly - MD's work on molecular diagnostics and point-of-care testing in the IVD thrust has implications for the effective and convenient diagnosis of old age diseases. Similarly, MD's work on cognitive care and remediation may have implications for the elderly (e.g., managed care implications and triage).

What is GBA+?

"GBA+ is an analytical process used to assess how diverse groups of women, men and non-binary people may experience policies, programs and initiatives. The "plus" in GBA+ acknowledges that GBA goes beyond biological (sex) and socio-cultural (gender) differences. We all have multiple identity factors that intersect to make us who we are; GBA+ also considers many other identity factors, like race, ethnicity, religion, age, and mental or physical disability." —Department for Women & Gender Equality

Stakeholder engagement

Overall finding: MD has worked with appropriate stakeholders to date. MD can increase its outreach and communication efforts for greater visibility as well as work more with pharmaceutical companies, major diagnostic firms and community health organizations. This would provide access to larger markets for research and the potential for high volume and greater impacts.

Sources: Case studies, internal and external interviews, document/literature review, data review, peer review.

Clients and collaborators

MD worked with a number of different types of clients and collaborators, of which the mix is appropriate given its objectives.

MD seeks to catalyze Canada's medical device industry, both by providing research and technology solutions to Canadian SMEs and MNEs with significant investment in Canada, and by working with OGDs to address Government of Canada priorities. The mix of MD's 112 clients is in line with this strategic priority.

The large proportion of industry clients to OGDs is also appropriate, and reflects the relevance of MD within the medical technologies industry ecosystem. Between 2012-13 and 2017-18, the source of MD's revenue shifted from a relative balance of OGD, academia and industry to predominantly industry (81% of revenues in 2017-18 were from industry). Likewise, 60% of MD's clients/collaborators belonged to industry.

MD worked with many types of clients/collaborators, with more than half being from industry (2012-13 to 2017-18)
Organizations
Long description of the pie chart
  Client share
Academia 14%
Other organizations 21%
Canadian OGDs 4%
MNEs 14%
SMEs (CND) 37%
SMEs (Foreign) 9%

*Other organizations include hospitals/research institutes, foreign government and non-profit. 50% were hospitals/research institutes.

The amount of revenue MD generated per employee between 2015-16 and 2017-18 was higher than that for other research centres within the NRC's Life Sciences Division (average of $50K per FTE compared to $40K for Life Sciences Division).

Future opportunities in stakeholder engagement

There are additional clients and collaborators that MD could target in order to have a greater impact on the Canadian economy.

MD's work with its current clients and collaborators contributed to its ability to expand its engagement with new organizations and in new areas (e.g., neurosurgery in the cognitive care industry).

In addition, some of the research centre's work resulted in opportunities to increase awareness of MD's capabilities (e.g., use of MD technology at demonstrations provided opportunities for visibility and awareness of its capabilities).

There are additional engagement opportunities for MD to consider, with larger markets for research and the potential for high volume and greater impacts. These include:

  • pharmaceutical industry, particularly if MD focuses its Simulation and Digital Health thrust on digital therapeutics, and contactless and visual monitoring of bodily signals/vital signs/data stream analysis
  • major diagnostics firms, for MD's In Vitro Diagnostics to further exploit its microfluidics and lab-on-a-chip technologies
  • community health organizations, including Indigenous health organizations, which would provide access to large patient populations to facilitate consideration of end users

Awareness of MD

Outside of its specific clients/collaborators, MD is not well known within the broader medical technologies ecosystem.

Barriers

  • Limited resources dedicated to business development, outreach and external promotion of MD's work (one Business Advisor, plus MD DG and two Section Heads)
  • Confidential nature of some projects that prevents use in presentations/publications
  • Limited ability to publish research (due to the NRC context between 2012-13 and 2016-17, where publishing was not viewed as a priority)
  • Restrictive government communications policies, and minimal/limited web presence

Recent improvements

  • As of November 2018, MD hired dedicated Client Relationship Leaders for each of its 3 thrusts.
  • Consistent with NRC-wide changes in 2017-18, there is an increasing emphasis on publications and conference presentations

Opportunities

  • Access more resources who are experienced in and dedicated to business development and stakeholder engagement (particularly with MNEs and other top players)
  • Embed entrepreneurs within each thrust that can go beyond technology transfer and contract development roles
  • Promote more of MD's scientific excellence via publications and participation in conferences, workshops and seminars
  • Identify innovative communications strategies, within the government's policies, and use a diversity of tools and instruments (e.g., annual reports that highlight MD's performance)

Capabilities

Overall finding: MD had the appropriate capacities, competencies and facilities to meet its objectives. Heavy dependence on key staff poses a risk to MD's ability to succeed, should these individuals leave. There are opportunities for MD to grow its operations, and play an even greater role in the medical device field. MD will need to make investments in its current facilities to remain state-of-the-art, which is necessary to continue its research in the future.

Sources: Data review, document/literature review, internal interviews, peer review

Capacities and competencies: Current state

MD had appropriate capacities and competences for where it was focused. The complement of staff have a proven track record. Execution of MD's strategy is heavily dependent on Section Heads and key resources, presenting a risk to the research centre.

NRC-wide risk – Ability to hire, retain, and train HQP

Demonstrated flexibility despite institutional constraints
MD demonstrated relative flexibility with its staff despite constraints, including

  • fewer staff than planned for due to NRC-wide hiring restrictions
  • having to build on expertise existing from prior NRC institutes
  • retraining some staff from the disbanded imaging thrust (In Vivo Imaging)
  • meeting the Health Technologies program objectives while MD staff were also tasked to work on other NRC non-MD programs (e.g., the Printable Electronics program and multiple Aerospace programs).

MD trained staff through internal R&D projects, and by providing staff time for professional development. In 2016-17 and 2017-18, 11% of staff time was on professional development, close to its target of 15%.

Appropriate capacity and competency
MD has appropriate capacities and competencies for the areas where they are focussed.

MD has an excellent staff composition according to the PRC. Several staff have a level of expertise that is at an internationally recognized level, namely in the In Vitro Diagnostics thrust with its current focus on microfluidics. Others are recognized nationally.

Close to half of MD's staff have a PhD. As a result, MD could leverage these staff to have a greater number of adjunct professorships. This would increase MD's ability to draw on students.

Human resource planning
MD has a dependence on Section Heads, with no clear succession plan. This presents a risk to the research centre. Their absence could affect the relevance of MD and its ability to achieve its objectives.

MD's 2019-20 Operational Plan includes the development of a Human Resource Plan. As MD works to develop and execute this plan, it will be important for it to ensure that it has a succession plan for the management team and key resources.

MD's exposure to retirement risk of its workforce is minimal, as MD employees are relatively young compared to the NRC overall. Most MD employees (61%) are aged 45 or younger. Many MD staff highlighted the same two reasons as to why people want to stay at the NRC, despite opportunities in the private sector:

  1. Interesting work
  2. Flexibility for work-life balance

Capacities and competencies: Growth opportunities

There is an opportunity for MD to grow its operations to have a greater impact in the medical device field.

Additional capacity and competencies
MD needs to add competencies to its staff in order to keep pace with the changing landscape of the medical devices sector and exploit opportunities. According to the PRC, failure to grow is both a lost opportunity and a prescription for eventual obsolescence. In order to exploit new opportunities in all three of its thrusts, additional resources would be needed.

This is particularly the case for the Implantable Devices thrust. While impacts to date have been appropriate given its size, MD can play a bigger role in the field of implantable devices if it were larger and had an expanded mandate.

If Simulation and Digital Health was to focus on digital therapeutics, and contactless and visual monitoring, additional expertise in these areas would need to be developed.

MD itself identified several competency gaps as important going forward. The In Vitro Diagnostics thrust identified gaps in bio-engineering and molecular data analysis, and the Simulation and Digital Health thrust identified gaps in mechatronics, programmers with sector-relevant knowledge (e.g., neuroscience and cognitive fields) and social media expertise to deploy its technology platforms on mobile devices. In addition, increased Technical Officers would allow for quicker turn around on proof of concept and support for maintenance of aging equipment.

Strategies to grow
With the current requirement to maintain or reduce operational expenditures across the NRC, MD's ability to grow is challenging. Collaborations with industry and academia, as well as the collaboration centres, will allow MD to expand its capacity and competencies. The use of students and post-doctoral fellows will also facilitate growth.

MD is also in the process of seeking options to transfer staff with misaligned competencies (due to the closure of a thrust, In Vivo Devices) to other NRC research centres, which will enable MD to hire additional resources with needed competencies.

Facilities

MD's R&D facilities are well organized and appropriate for its needs. However, the equipment in MD's major facility, the BioAnalytical Micro-Devices Clean Room, is in need of renewal to remain state of the art. There is a potential to grow MD's facilities to address promising opportunities.

NRC-wide risk – Aging infrastructure

Capital investments
MD faced challenges accessing capital investment (minor and major) for facility improvements. This is partly due to the elimination of 2 research thrusts and the associated planned capital expenditures, as well as limited spending on major capital (due to the fact MD has not been as successful against other NRC research centres in accessing a limited pool of major capital funding).

MD did not access capital investments needed for aging facilities (2015-16 to 2017-18)
Capital investments
Long description of the horizontal band chart

Planned $5,200,000

Actual $2,700,000

Appropriateness of facilities
The In Vitro Diagnostics thrust generally has the equipment it needs within its BioAnalytical Micro-Devices Clean Room. The age of some equipment (i.e., 10-15 years old) is a concern and upgrades are needed to remain state of the art and relevant to stakeholders. The age of some equipment (i.e., 10-15 years old) is a concern and upgrades are needed to remain state of the art and relevant to stakeholders. In 2017-18, the infinity double side mask aligner imploded, which sent Hg vapour into the lab. This safety risk is, however, being mitigated by the research centre. Newer system use laser illumination instead of Hg lamps. The high volume hot embossing system, purchased in 2009-10, was one of the key pieces of equipment that differentiated the NRC from others in the field.

The In Vitro Diagnostics thrust will be able to access newer facilities and equipment through the collaboration centres with the University of Toronto and CHU Sainte-Justine. According to the PRC there would also be significant return on investment for investments in an in vitro diagnostics research and manufacturing hub, which could be achieved through a co-location model of academia, industry, and the NRC in a single building.

The equipment for the Simulation and Digital Health thrust is current and state of the art. It may need more space but this can be achieved through NRC locations in Vancouver that are supporting the Digital Technology Supercluster.

The Implantable Devices thrust is supported by the Automotive and Surface Transportation Research Centre's Powder Metallurgy facilities. The PRC found these to be state of the art and appropriate for the thrust's needs.

Performance

Overall finding: MD's research was leading edge and contributed to advancements in the medical device field. It also had a positive impact on clients, supporting business innovation and growth of the medical device industry. MD's research is positioned to support government policy solutions in the future.

Sources: Case studies, peer review, bibliometric study, data review, document/literature review, internal interviews

Scientific impact of MD's published research

The scientific impact of MD's published research within academia was not as high as that of the NRC or Canada overall. MD's published research, however, had more of an applied role and was used downstream in patents, medical and treatment guidelines, health policies, and health insurance assessments.

Impact within academia
MD publications between 2012 and 2017 were not cited as much as the NRC and Canadian publications (see Appendix D for data by research thrust).

Field Weighted Citation Index, 2012 to 2017
Field Weighted Citation Index
Long description of Field Weighted Citation Index, 2012 to 2017
  Field Weighted Citation Index
Canada 1.51
NRC 1.47
MD 1.05

Impact beyond academia
MD's publications between 2012 and 2017 were cited in more patent documents than the rate for the NRC or Canadian publications. This indicates they are being used in applied work.

Likewise, between 2011-2017, 38 (out of 326) of MD's publications were used outside of academic publications. 3.7% were referenced in Medical guidelines, health benefit programs, or clinical trials, and 8.6% were cited in patent documents. The proportion of MD publications cited in patents, and decision-making and policy-setting documents was similar to that of the Canadian Institute for Health Research (between 2011-2013, 15.2% of MD's publications were cited beyond academia compared to 18% for the Canadian Institute for Health Research). Note: time periods used differ due to methodological considerations; see methodology appendix for further details.

Patent Citation Rate per 1,000 Publications, 2012 to 2017
Patent Citation Rate
Long description of Patent Citation Rate per 1,000 Publications, 2012 to 2017
  Patent Citation Rate per 1,000 Publications
Canada 19.4
NRC 46.8
MD 63.4

Advancement of scientific knowledge

MD's work was innovative and contributed to advancements in the medical device field. Its current microfluidics and lab-on-a-chip technologies as well as its previous neurosurgical surgical simulators are of international stature.

Innovations at a glance (2012-13 to 2017-18)
Innovations at a glance
Long description of Innovations at a glance (2012-13 to 2017-18)

A white circle is in the middle of 3 blue circles. The white circle, labeled Innovations at a glance (2012-13 to 2017-18) connects with the 3 blue circles.

  • 0.6 inventions disclosed per staff, higher than the NRC Life Sciences Division (at 0.4)
  • 13 unique patent applications filed
  • 2 unique patents granted of the 13 applications

International stature
MD's current research in microfluidics and lab-on-a-chip are internationally recognized, as well as its previous work on neurosurgical simulators (i.e., NeuroVR).

If MD takes a more strategic direction within its Simulation and Digital Heath and Implantable Device thrusts, refocusing around areas of strength, it has the potential to reach international excellence in these areas as well.

Innovations
MD has an impressive intellectual property portfolio, particularly in microfluidics. Per capita, MD also had the highest rate of innovations within the Life Sciences Division at the NRC.

According to the PRC, the number of unique patents filed and held by MD, as well as its annual intellectual property targets (5 patents per year) could be greater than is currently the case. NRC-wide factors, however, need to be taken into consideration. The patenting process at the NRC may have inhibited researchers willingness to invent. There currently is an NRC initiative to review the inventor award policy and incentive structure.

Advancements to the medical device field
Implantable Device's foam technology led to projects focused on the same technology for use in human health applications.

Simulation and Digital Health's haptic research has helped pushed research in that area forward

In Vitro Diagnostics centrifuge-based assay and use of thermoplastic elastomer work are significant enhancements for that field of science.

Business innovations impacts

MD's work with its clients and collaborators resulted in positive outcomes that will contribute to the growth of the medical device industry, including transferred knowledge and technology, as well as increased product valuation, commercialization and market valuation.

Immediate outcomes
(2012-13 to 2016-17)

Transferred knowledge and technologies
Provided technical expertise and strategic guidance needed by SMEs to take risks in undeveloped but emerging and promising areas

Signed licensing agreements with MNEs

Transferred knowledge and know-how to SMEs on how to produce and replicate new devices, and supported manufacturing scale-up

Intermediate outcomes
(2017-18 to 2021-22)

Increased commercialization
All 3 thrusts have led clients launching new medical device products

Increased market valuation
Clients reported growth in revenues, expanded capacity, and strengthened position within the sector

Increased product valuations
Collaborations connected clients to new business opportunities

Long-term outcomes
(2022-23 and beyond)

Cost-effective technological solutions
Simulators expected to reduce costs associated with risks and complications of neurosurgery, as well as training costs

Improved health of Canadians
Simulators have potential to reduce training time, help more surgeons develop life-saving skills and minimize risks and complications

Government solutions

MD is positioned to contribute to government policy solutions, as demonstrated by the alignment of its work with key areas in the areas of health and bio-sciences, and medical devices research and innovation as well as its work with other government departments.

Alignment with federal government policies and priorities
Digital health (e.g., medical device connectivity) is one priority areas in the Health/Bio-sciences Strategy of the Innovation and Skills Plan.

Health technology commercialization of is a priority of the Build in Canada Innovation Program.

MD will support the Digital Technology Supercluster, aimed at extending digital technology capabilities to an increasing number of industry sectors, including health.

MD's recent move toward focusing on a remote, interactive healthcare platform is aligned with federal priorities on Indigenous Nations and other rural/remote populations.

The large majority of MD's publications are aligned with Government of Canada's priority areas
Long description of the graph on MD publications
  Series 1
Health/biosciences 86%
Digital industries 38%
Advanced manufacturing 38%
Agri-food 16%
Clean technologies 3%
Resources for the future 1%
Not in priority areas 4%

Past and ongoing collaboration with federal departments with future impacts on health and safety Health Canada/Canadian Food Inspection Agency: Via its In Vitro Diagnostics thrust, MD worked with Health Canada to produce new tools capable of monitoring in real-time presence of Listeria monocytogenes as well as other foodborne pathogens using MD's lab-on-a-chip technology. The Canadian Food Inspection Agency is the end user of this work, and will be able to identify pathogens in contaminated food within 8 hours instead of 5 days. This is to be implemented over the next 5 years.

Communications Security Establishment: Via its Simulation and Digital Health thrust, MD is working with CSE on a whitepaper "Cybersecurity for Medical Devices: Recommended Best Practices During Design, Development and Deployment".

MD has also worked with Defense Research and Development Canada (via the disbanded In Vivo Imaging thrust), and is embarking on a new R&D project with the Canadian Space Agency (via In Vitro Diagnostics).

Conclusions and recommendations

Relevance
MD is addressing a need in the medical device field. While MD's research has focused on areas of importance to the medical device industry, as well as addressed issues of societal value, MD can make adjustments to its research strategy as it goes forward. This includes exploiting its microfluidics and lab-on-a-chip technologies to support the development of a new industry in Canada, expanding the vision/mandate of its Implantable Devices thrust, and refining the focus of the Simulation and Digital Health thrust to digital therapeutics and contactless/visual monitoring of bodily signals. MD's potential is far greater than what is currently realized, and would require MD to grow.

Capacity
While MD's competencies and facilities were appropriate to date, there is an opportunity for MD to grow to have an even greater impact. In addition, key equipment must be renewed to remain state-of-the-art. This is key to MD's relevance and ability to meet stakeholder needs. Over the evaluation period, MD had limited resources dedicated to business development, outreach and external promotion of its work. MD now has additional business development staff. MD's reliance on its Section Heads poses a risk to MD, should these individuals leave. MD could also capitalize on the high educational attainment of its staff by increasing the number that hold adjunct professorships at universities. This would increase access to more students and post-doctoral fellows.

Performance
MD worked with many clients and collaborators. Clients and collaborators reported that MD contributed to the growth of the medical device industry, including increased product valuation, commercialization and market valuation. MD can make itself more known in the medical device ecosystem and expand its stakeholder engagement.

In addition to increased business innovation and positive impacts on clients, MD has contributed to advancements of scientific knowledge in the medical device field. MD's work was innovative and scientifically excellent, attaining international stature in some areas (microfluidics, lab-on-a-chip, neurosurgical simulators). Its publications were cited in patents, treatment guidelines, and health policies. If MD refocuses its Simulation and Digital Health thrust and its Implantable Devices thrust, it can reach international status in these areas in the future.

Finally, MD is positioned to contribute to government policy solutions, as demonstrated by the alignment of its work with federal government priorities as well as its work with other departments.

Recommendations

Recommendation 1
MD should devise and implement a strategic planning process for each of its three research thrusts, and update these plans on an annual basis. Plans should identify opportunities to exploit (e.g., In Vitro Diagnostics' lab-on-a-chip technology), reconsider and revise research focus where appropriate (e.g., Simulation and Digital Health, and Implantable Devices), and consider involvement of and implications for end-users (e.g., patients, diverse populations).

Rationale: Only one of MD's 3 research thrusts has a long-term vision. The other 2 thrusts would benefit from re-examining their research strategies, which are more reactive and address incremental contributions to the field. Contributing to this was the absence of a clearly defined strategic planning process, including consultation and consideration of some end-users such as patients.

Recommendation 2
MD should develop stakeholder engagement plans, per research thrust, and report against progress made to ensure continued alignment with the strategic plan. As part of these stakeholder engagement plans, consideration could be given to the pharmaceutical industry, major diagnostics firms and community health organizations. Engagement plans should be revisited on an annual basis to maintain alignment with the above-recommended strategic plans.

Rationale: MD has not maintained its stakeholder engagement plans. The maintenance and monitoring of stakeholder engagement plans will help ensure that MD's outreach efforts support its strategic direction. Stakeholder engagement plans will also contribute to the recommended strategic planning process by structuring outreach efforts to identify and prioritize industry needs.

Recommendation 3
MD should identify strategies to increase awareness of its capabilities within relevant industries, in particular those where its profile is low but there are opportunities for growth (e.g., pharmaceutical industry, major diagnostics firms).

Rationale: Outside of its current clients and collaborators, MD is not well known. As a result of this, there is a risk that MD is unable to reach key stakeholders.

Recommendation 4
MD should develop a strategic plan for staff development and succession planning for each of its thrusts.

Rationale: MD is highly dependent on its Section Heads. This presents a risk to the research centre's ability to achieve its objectives, should these individuals leave. MD also has a large number of staff with advanced degrees that could be leveraged for greater access to students and post-doctoral students (e.g., through adjunct professor status at universities).

Recommendation 5
MD should prioritize investments in its major facility for In Vitro Diagnostics, the Bio Analytical Clean Room, to support current work and to allow for future growth.

Rationale: The equipment in MD's Clean Room may no longer be considered state of the art. Key components, which once distinguished MD from other organizations, are now outdated (in some cases 10 to 15years old). The aging equipment requires renewal to conduct its current work in the future and to maintain its leading-edge status in microfluidics.

Management Response and Action Plan

Recommendation Risk-level Associated with not Addressing Recommendation Management Response Measure of Achievements Proposed Person(s) Responsible Expected Date of Completion

1. MD should devise and implement a strategic planning process for each of its three research thrusts, and update these plans on an annual basis. Plans should identify opportunities to exploit (e.g., In Vitro Diagnostics' lab-on-a-chip technology), reconsider and revise research focus where appropriate (e.g., Simulation and Digital Health, and Implantable Devices), and consider involvement of and implications for end-users (e.g., patients, diverse populations).

High

Response: Accepted
Action 1: Hire Strategic Advisor responsible for, among other things, establishing process and template for Research Thrust Strategic Plans.
Action 2: Develop strategic planning process and templates.
Action 3: Draft Research Thrust Strategic Plans that are based on a systematic process with consideration of and consultation with stakeholders and end-users like patients and / or diverse populations.

  • Advisor hired.
  • Strategic planning process and templates developed.
  • Individualized plans for MD Research Thrusts.

 

Director General, MD

March 2020

2. MD should develop stakeholder engagement plans, per research thrust, and report against progress made to ensure continued alignment with the strategic plan. As part of these stakeholder engagement plans, consideration could be given to the pharmaceutical industry, major diagnostics firms and community health organizations. Engagement plans should be revisited on an annual basis to maintain alignment with the above-recommended strategic plans.

Medium

Response: Accepted
Action 1: Draft engagement plans for each thrust and establish a renewal framework.

  • Stakeholder engagement plans for each Research Thrust.

 

Director General, MD

June 2020

Action 2: Report against progress in engagement plans for each thrust.

  • At least 80% delivery on action items in the stakeholder engagement plans.

 

June 2021

3. MD should identify strategies to increase awareness of its capabilities within relevant industries, in particular those where its profile is low but there are opportunities for growth (e.g., pharmaceutical industry, major diagnostics firms).

Medium

Response: Accepted
Action 1: Draft marketing plan to increase awareness of MD.

  • Marketing plan in place and being implemented.

 

Director General, MD

May 2020

4. MD should develop a strategic plan for staff development and succession planning for each of its thrusts.

High

Response: Accepted
Action 1: Develop staff development plans for succession planning.

  • Professional development plan outlined for High potential personnel (HIPO).
  • Initial meetings with individuals identified in the plan to discuss development plans.

Director General, MD

May 2020

5. MD should prioritize investments in its major facility for In Vitro Diagnostics, the Bio Analytical Clean Room, to support current work and to allow for future growth.

High

Response: Accepted
Action 1: MD will explore options for increased investment, internally and externally, and seek NRC Senior Executive approval.

  • Investment options identified.
  • Investment options presented to SEC and/or other group, as appropriate.

Director of Operations, MD

June 2020

Appendices

Appendix A – Methodology

Bibliometric study
The National Science Library's Intelligence and Analytics team conducted a bibliometric assessment of MD's peer-reviewed publications indexed in Scopus for the period 2012–2018 to assess scientific excellence and impact (both within academia and beyond academia). In addition, the NRC's National Science Library collaborated with the Canadian Institute for Health Research to draw on the database they created to assess the use of academic publications beyond academia (e.g., in Medical and treatment guidelines, health policies, health insurance assessments). When looking at impact beyond academia, publications from 2011 were included as it takes time for these publications to have an influence beyond academia (e.g., 7 – 10 years). The time period for comparison with the Canadian Institute for Health Research was limited to 2011-2013 because that was when data was available.

Case studies
In depth case studies were conducted on the impacts of MD's work with six clients. These case studies were chosen to be high impact cases and represented MD's research thrusts, project types, client types and project sizes. The case studies included a review of project documents as well interviews with the MD project lead (s) and representative (s) from the client organization. In total, 16 interviews were conducted - eight with clients and eight with MD project leads. Overall, the case studies covered 12% of MD projects between 2012-13 and 2016-17 and 29% of MD's revenue.

Document and literature review
Internal and external documents were reviewed to provide context and to complement other lines of evidence in assessing relevance and performance.

Data review
Research Centre and program administrative and performance data for 2012-13 to 2016-17 were reviewed to provide information on program inputs (i.e., resources), outputs, and client reach. This included financial data, human resource data, project data and intellectual property data.

Key informant interviews
Interviews were conducted with 25 stakeholders (19 internal and 6 external) to collect information such as personal experiences, opinions and expert knowledge related to the relevance and performance of MD. This information was used to complement other lines of evidence and to contextualize quantitative information.

Peer review
A peer review committee was convened to assess MD along four dimensions: relevance, stakeholder engagement, performance and appropriateness of resources, including capabilities. The Committee was composed of three members plus one chair and included individuals with expertise related to in vitro diagnostics, simulation and digital health, and implantable devices. Members included national and international representatives from academia, government, and industry. Members were expected to participate in the review process in an objective, unbiased and credible manner, with no apparent or perceived conflict of interest. To this end, all members signed a confidentiality and conflict of interest agreement. Each peer review process included:

  1. reviewing background material produced by the program and by the NRC evaluation team
  2. participating in a pre-site visit teleconference to discuss the Committee's initial assessment of the programs, information gaps and questions.
  3. participating in a two and a half day site visit to the NRC

Limitation and mitigation strategies

Availability of data and documents
Valid and reliable data on facility use was not available. As such, the evaluation was not able to assess facility use. In the last two years, MD has, however, begun the process to do so. Likewise, up to date stakeholder engagement plans were not maintained and outreach activities not tracked. As a result, it was difficult to determine whether MD had reached appropriate stakeholders. The views of the peer review committee were used to inform the extent to which MD had reached the right stakeholders.

As a result of changes to MD's Health Technologies program, updated program documentation (e.g., business plan) did not exist. This made it challenging to assess MD's performance against its identified objectives and plans. In order to mitigate this challenge, interviews with internal staff were used to compose an updated profile of MD.

Use of publications to measure excellence
The challenge with bibliometric analysis is that there is a time lag of citation of published work. As a result, the actual use of more recent publications is likely underestimated in the current study. Additionally, not all of MD's research areas publish at the same rate – research in the Simulation and Digital Health thrust is published less given the field it is in - as in computer sciences/software, for which publishing is not the norm. To mitigate this limitation, other lines of evidence were used to assess the excellence and scientific impact of MD's research as well.

Representativeness of case studies
Clients chosen for case studies represented what were expected to demonstrate high impact. There is the possibility that this does not accurately reflect the true impact of the research centre. This, however, is mitigated by the fact that the peer review committee will be exposed to a wider array of MD's work, and will be in a position to comment on whether MD's overall performance is similar to what was observed through the six case studies.

Peer Review Committee members

Mehran Anvari, Chair: Professor, McMaster University; Scientific Director and CEO, Centre for Surgical Invention & Innovation (CSii); Editor-in-Chief, International Journal of Medical Robotics and Computer-Assisted Surgery

Héloïse Côté: Trade Commissioner, Medical Technologies, Global Affairs Canada

Donald Jones: Chairman, Cardiff Ocean Group LLC; Chief Digital Officer, Scripps Translational Science Institute; Advisor/Advisory Board Member, health-related companies and multiple Fortune 1000 Digital Health Advisory Boards

Michael V. Sefton: Professor, University of Toronto; Executive Director, Medicine by Design

Appendix B – List of key documents reviewed

Appendix C – Organizational chart

Organization chart
Long description of the Organizational chart

Organisational chart for Medical Devices research centre

  • Vice President, Life Sciences
    • Director General, Medical Devices
      • Operational and Project Management (4 managers and administrative staff, 3 research officers)
      • BioAnalytical Micro Nano Devices (18 research officers, 18 technical officers)
        • Research thrust: In vitro Diagnostics
      • Simulation and Digital Health (18 research officers, 7 technical officers)
        • Research thrust: Simulation and Digital Health
    • Automotive Surface Transportation Research Centre
      • 2 full time equivalents (also supported by two MD resources)
        • Research thrust: Implantable Devices