Status: | Active |
---|---|
Effective date: | June 26, 2020 |
Prepared by: | Office of Audit and Evaluation, National Research Council Canada |
Approval: | NRC's President |
Related document: | Summary - Evaluation of the Automotive and Surface Transportation Research Centre |
ISBN: | 978‑0‑660‑35452‑1 |
Alternate format: | Evaluation of the Automotive and Surface Transportation Research Centre (PDF, 1.3 MB) |
Acronymes
- AMP
- Advanced Manufacturing Program
- AERO
- Aerospace Research Centre
- AST
- Automotive and Surface Transportation Research Centre
- ATC
- Aluminum Technology Centre (AST site)
- ATSP
- Advanced Transportation System Program
- CAV/ITS
- Connected and autonomous vehicles / Intelligent transportation systems
- CSTT
- Centre for Surface Transportation Technology (AST predecessor, transportation‑side)
- DND
- Department of National Defence
- DT
- Digital Technologies Research Centre
- ECCC
- Environment and Climate Change Canada
- EME
- Energy, Mining and Environment Research Centre
- FF2020
- Fleet Forward 2020 (program)
- FTE
- Full‑time equivalent
- FWCI
- Field‑Weighted Citation Index
- GBA+
- Gender‑based analysis plus
- GHG
- Greenhouse gas
- HPC
- High performance computing
- IMI
- Industrial Materials Institute (AST predecessor, manufacturing‑side)
- IRAP
- Industrial Research Assistance Program
- IP
- Intellectual property
- MNE
- Multinational enterprise
- NGen
- Next Generation Manufacturing cluster
- NRC
- National Research Council of Canada
- NRCan
- Natural Resources Canada
- OAE
- Office of Audit and Evaluation
- OEM
- Original equipment manufacturer
- OGD
- Other government department (Canadian federal)
- PERD
- Program for Energy Research and Development (NRCan program)
- PRC
- Peer review committee
- RC
- Research centre
- RTO
- Research and technology organization
- RVTO
- Rail Vehicle and Track Optimization (program)
- SME
- small and medium‑sized enterprise
- TC
- Transport Canada
- TRL
- Technology readiness level
- VPT
- Vehicle Propulsion Technologies (program)
Executive Summary
The Automotive and Surface Transportation (AST) Research Centre supports the advanced manufacturing and ground transportation sectors to identify, develop and apply relevant science and technology that will ensure Canada’s continued prosperity through revolutions in these sectors. It was established in 2013‑14 when NRC merged the automotive portfolio of the Industrial Materials Institute (IMI) and the Centre for Surface Transportation Technology (CSTT). AST’s strategic research and development (R&D) and technical services activities may be divided into two sections: material processing and manufacturing, and vehicle and transportation systems engineering.
The evaluation covered 2013‑14 to 2018‑19 and is the first to evaluate AST as a single, integrated research centre. Lines of evidence included a bibliometric study, client survey, data review, document/literature review, external and internal interviews, and a peer review by experts from industry and academia. The key findings and recommendations of the evaluation were as follows:
Relevance
AST is differentiated by its breadth of research (offering it a competitive advantage), as well as its recognized expertise and large‑scale facilities. As such, it stands out as unique in Canada, though not internationally. Its research is aligned with the priorities of industry and of the Government of Canada government, but project selection has tended to be more reactive than strategic, and AST has yet to fully exploit its competitive advantage.
Engagement
AST has employed a variety of engagement strategies, including a successful R&D group model for sectoral engagement. Overall, visibility among and engagement with stakeholders could improve, particularly with the research community in order to effectively implement its proposed Advanced Transportation Systems Program (ATSP).
Resources
AST has had sufficient resources to meet client needs to date, but lacks the critical mass of expertise and facilities needed to move into new and emerging areas.
Scientific excellence and leadership
AST has advanced scientific and technical knowledge in material processing and manufacturing, and in niche areas of transportation engineering (e.g., rail) through technology deployment and optimization.
Impact
AST’s work in both the advanced manufacturing and ground transportation sectors is contributing to economic, environmental, and social impacts, as well government policy solutions.
Recommendation 1: AST should articulate a plan that clarifies and promotes the integration of its capabilities in both material processing and manufacturing, and vehicle and transportation systems engineering, and pilot strategies to support this integration. This plan should identify where AST wants to be a leader (and commit resources accordingly), opportunities and risks for Canada posed by mutual disruptions to the advanced manufacturing and ground transportation sectors, and how it will contribute to Canada’s competitive advantage in these sectors.
Recommendation 2: AST should increase awareness among and engagement with key stakeholders in new and emerging fields. In particular, it should diversify and broaden its customer base in vehicle and transportation systems engineering, including greater collaboration with key players such as universities in order to access their competencies and facilities.
Recommendation 3: AST should identify and implement strategies to address talent and facility gaps, particularly for its work related to advanced transportation systems. In addition to operationalizing existing HR and facility strategies, AST should explore new strategies to increase capacity to ensure its successful entry into new fields.
Recommendation 4: AST should determine and communicate strategic indicators that measure its contribution to the Canadian economy, demonstrate scientific excellence (using more industry‑relevant metrics), and track influence on government operations, policy, and regulations.
Introduction
An evaluation of the AST Research Centre (RC) was conducted in 2019‑20, in accordance with the NRC’s approved evaluation plan and Treasury Board policies. The evaluation covered the years 2013‑14 to 2018‑19 and is the first to evaluate AST as a single, integrated research centre.
This report begins by providing a profile of AST. It then presents the evaluation findings on AST’s relevance, stakeholder engagement, resources, scientific excellence, and impact. Following the conclusion are 4 recommendations for improvements within AST.
Evaluation approach
Evaluation questions
The evaluation questions were developed based on consultations and a review of key documents, and guided the conduct of the evaluation.
- Is AST focused in appropriate areas to ensure relevance of the research centre?
- Has AST engaged the most appropriate clients and collaborators in the most effective ways?
- To what extent does AST have the capacities, competencies, and facilities to achieve its objectives?
- To what extent was AST a leader in scientific excellence in the areas of ground transportation R&D (including vehicle testing) and advanced manufacturing?
- to what extent is AST positioned to be a leader in advanced ground transportation systems in the future?
- To what extent has AST contributed to longer term outcomes, that is:
- the economic growth and prosperity of the Canadian transportation and manufacturing industries?
- government operations, policy and regulations?
- social and environmental impacts?
Research methods
Mixed methods were used to maximize the generation of useful, valid and relevant evaluation findings. This approach also allowed for cross‑validation of results based on the following methods:
- bibliometric study (publication and patent citation analysis)
- client survey (n=128, response rate of 28%)
- data review (administrative and performance data)
- document/literature review
- external (n=21) and internal interviews (n=31)
- peer review by committee of nine experts from academia and industry
See Appendix A for a more detailed methodology including challenges and limitations.
Profile
AST’s vision is to make Canada more competitive through innovations in mobility and manufacturing excellence. It supports several industries including advanced manufacturing, automotive, defence and security, rail, and trucks and buses.
Programs and services
AST’s vision is to make Canada more competitive through innovations in mobility and manufacturing excellence. Its activities may be divided into two sections: material processing and manufacturing; and, vehicle and transportation systems engineering. AST’s programs mostly align with one section or the other, though one aligns with both (VPT). AST identifies, develops, and applies relevant science and technology that will ensure Canada’s continued prosperity through the revolutions in the advanced manufacturing and ground transportation sectors.
Material processing and manufacturing
This activity focuses on smart and digital manufacturing, and advanced materials and manufacturing processes. It includes one program of its own, the Advanced Manufacturing Program (AMP), which aims to help position Canada as a leader in smart manufacturing. AST’s work in this area was previously organized in three programs (Industrial Biomaterials, Light‑weighting of Ground Transportation Vehicles and Advanced Manufacturing and Design Systems), which have now been merged to form the AMP. The AMP supports Canada’s Next Generation Manufacturing (NGen) cluster, a federal initiative to encourage the development of regional ecosystems to create jobs and support innovation.
Cross‑cutting programming
Crossing both of AST’s sections, the Vehicle Propulsion Technologies (VPT) aims to deliver propulsion technology and increase fuel economy. Though most VPT projects have fit under the materials and manufacturing section (e.g., lithium battery, hydrogen fuel cells, and low cost electric motors manufacturing), it has shifted over time to include more transportation engineering projects (e.g., battery safety and performance in the Canadian context) to provide evidence for larger adoption of electric vehicles. For analysis and reporting purposes, this program was split at a per‑project level to allow for accurate comparisons between sections.
Vehicle and transportation systems engineering
This activity addresses advanced vehicle integration, rail vehicle design, testing and evaluation, dynamic modeling and simulation, wheel/rail interaction, vehicle testing, and connected and autonomous vehicles/intelligent transportation systems (CAV/ITS). It currently includes one service line and two programs:
- Defence and Security Service Line: provides a range of technical services to support the Department of National Defence (DND), the Royal Canadian Mounted Police (RCMP), and the Canadian defence industry.
- Fleet Forward 2020 (FF2020) Program: aims to improve the operational efficiency of targeted Canadian fleet transportation.
- Rail Vehicle and Track Optimization (RVTO) Program: aims to improve operational efficiency and safety of Canadian railways.
AST is developing a new program, the Advanced Transportation Systems Program (ATSP), which will contribute to the transformation of the Canadian surface transportation sector by developing a safer, efficient, more intelligent, more secure and sustainable transportation system. It is intended to launch in 2020‑21 and consolidate the RVTO, FF2020 and VPT programs. ATSP will include more CAV/ITS research with planned niche focusses in perception and mapping systems, and adapting technologies to the Canadian context (e.g., signage, conditions).
Why manufacturing and transportation? AST was created in April 2013 by merging the NRC’s surface transportation portfolio with its automotive portfolio under a single structure in order to enhance the management of resources and capabilities, and direct manufacturing activities towards the automotive and transportation industries.
Human resources
As of March 31, 2019, AST had a total of 278 staff, including 19 students, and 24 major facilities and other installations, spread across four sites in Quebec and Ontario. AST also draws on human resources and facilities from the NRC’s other RCs to complement its own capacities. In 2018‑19, other RCs contributed 47 full‑time equivalents (FTEs; defined as 1,450 hours of labour) to AST’s projects and programs. Most of these came from the Aerospace (AERO), Energy, Mining and Environment (EME), and Digital Technology (DT) research centres.
Each site acts as its own research directorate with a Director of R&D at the helm. The Boucherville site also hosts the Director General’s office (DG), Operations & Project Management offices, and other common services. The DG reports to the Vice‑President, Transportation and Manufacturing.
The Transportation Engineering Centre (see below) was previously the NRC’s Centre for Surface Transportation Technology (CSTT), which focused primarily on technical services and operated under a cost‑recovery model. The Saguenay, London, and Boucherville groups (right) previously belonged to the NRC’s Industrial Materials Institute (IMI), which was more focused on strategic R&D. When the IMI was disbanded in 2012‑13 into separate portfolios (including the automotive), other RCs were formed from it including EME and Medical Devices (MD), both of which continue to collaborate with AST staff.
Transportation Engineering Centre (TEC), Ottawa, ON
- 48 researchers and 43 technicians in a bilingual setting
- primarily focused on research and technical services in vehicle and transportation systems engineering
Aluminum Technology Centre (ATC), Saguenay, QC
- 16 researchers and 12 technicians in a francophone setting
- focused on materials/manufacturing (e.g., light weighting)
Manufacturing and Automotive Innovation Hub, London, ON
- 19 researchers and 7 technicians in an anglophone setting
- primarily focused on manufacturing, but contributes to some transportation engineering projects (e.g., CAV/ITS)
Materials & Processes, Boucherville, QC
- 60 researchers and 30 technicians in a francophone setting
- primarily focused on materials/manufacturing, with some exceptions (e.g., battery safety in transportation systems)
Facilities
AST’s facilities are located at four sites in Quebec and Ontario. It also accesses facilities maintained by other NRC RCs, such as AERO’s wind tunnels for testing vehicle aerodynamics and EME’s battery testing facilities.
ATC, Saguenay, QC
- industrial scale automated welding, forming, and casting laboratories
- scanning electronic microscope and metallography labs
- corrosion and surface treatment labs
- mechanical testing and virtual laboratory for numerical modeling and simulation of processes
Manufacturing and Automotive Innovation Hub, London, ON
- two experimental vehicle bays with lifts and related automotive shop capabilities
- high crane bay as a manufacturing space, and work cells including advanced automation, laser manufacturing and advanced machining for scale‑up and on‑site digital factory
Materials & Processes, Boucherville, QC
- advanced polymer matrix composites processing and prototyping at large scale
- lab and pilot scale processing and prototyping of composites and polymers
- high performance computing (HPC) cluster
- lab and pilot scale battery and fuel cell materials, components and cells fabrication and testing
- powder forming facilities
- thermal spray and cold spray facilities
TEC, Ottawa, ON
- railcar‑size climatic chamber (and small chambers)
- heavy vehicle structural dynamics lab
- rail testing facilities
- rail wheel bearing and brake facility
- vehicle field testing and instrumentation equipment (instrumented wheelsets, tilt‑table)
Facilities review: AST is also receiving guidance through the NRC‑wide Facilities Review. TEC facilities were reviewed concurrent to this evaluation, but other AST facilities reviews are still ongoing. As such, this evaluation has limited its facility‑specific findings as AST will be receiving more detailed findings from the review.
Revenues and expenses
Met technical service targets, but has missed R&D targets
AST sets annual revenue targets for both technical services and strategic R&D (see figure top right). It met or exceeded its technical service targets for the past four years but consistently missed its strategic R&D targets.
Transportation engineering recovering costs, materials/manufacturing focused on R&D
AST earned $149M in revenues in its first six years as a unified RC. Technical services accounted for 71% of revenues, while strategic R&D accounted for 27%. Other sources (2%) included grants and contributions, sale of goods and information products, and lease and use of property. These trends differed significantly by research area (see bottom right figure).
- transportation engineering revenues ($91M) mostly came from technical services (92%), reflective of this section’s pre‑AST cost‑recovery model.
- materials/manufacturing revenues ($51M) mostly came from R&D projects (65%), either for individual clients or its industrial R&D group members. This section also dedicated considerable time to internal R&D projects (no client), and gave in‑kind support for R&D group projects; hence its higher expenses.
AST also generated $7M in revenues from other technical service projects that it categorized outside of their transportation and manufacturing programs.
In total, AST had $300M in expenses, most of which (62%) were directly related to transportation engineering and materials/manufacturing projects. The rest (38%) in both its research areas a, facility and equipment maintenance, RC management, and administration.
Figure 1 ‑ Text version
Long description of Meeting technical service targets, missing strategic R&D targets
The figure is a vertical combination bar and line chart that depicts AST’s target vs. actual revenues for Technical Services and Strategic R&D projects, respectively. The data shows that while AST achieved its Technical Services revenue target in four out of six years, the Research Centre consistently fell short of its revenue target for Strategic R&D.
Fiscal Year | Technical Services: Target Revenue | Technical Services: Actual Revenue | Strategic R&D: Target Revenue | Strategic R&D: Actual Revenue |
---|---|---|---|---|
2013‑14 | $22M | $18M | $9M | $6M |
2014‑15 | $20M | $13M | $11M | $6M |
2015‑16 | $17M | $18M | $8M | $6M |
2016‑17 | $19M | $20M | $10M | $6M |
2017‑18 | $19M | $19M | $9M | $8M |
2018‑19 | $16M | $17M | $10M | $8M |
Figure 2 ‑ Text version
Long description of Different revenue priorities by research area (2013‑14 to 2018‑19)
The figure is a horizontal stacked bar chart that compares the total revenues generated by AST’s two major research areas: Materials/Manufacturing, and Transportation Engineering. Total revenues in each area are broken down by their source: Strategic R&D, Technical Services, or “other”. For reference, the total program expenses attributed to each research area are also depicted alongside the revenues.
Research Area | Technical Services Revenue | Strategic R&D Revenue | Other Revenue Sources | Total Program Expenses |
---|---|---|---|---|
Materials/Manufacturing | $16M | $33M | $2M | $94M |
Transportation Engineering | $84M | $7M | $0M | $88M |
Strategic R&D versus technical services? Strategic R&D, or research services, consists of collaborative research projects undertaken with partners to de‑risk R&D and accelerate commercial development timelines. Technical services consist of projects that assist clients in solving immediate technical problems through the delivery of specialized fee‑for‑service support (e.g., testing and certifications, calibration, prototyping, demonstrations, scale‑up, and consulting).
Clients and collaborators
AST envisions itself as a national convener, gathering transportation and manufacturing stakeholders across the value chain from academia, government, and industry to breach financial, capabilities, and technical obstacles through collaborative R&D. Between 2013‑14 and 2018‑19, AST worked with 510 unique organizations, on standalone projects and through its industrial R&D groups. Firstly, AST completed or launched over 1,250 standalone projects for 433 unique clients and collaborators. Secondly, it also completed or launched over 120 collaborative projects with its industrial R&D groups, which involved another 71 unique clients, collaborators, and sponsors (see more in the Engagement section, slides 22‑23). In total, the R&D groups had 109 members or sponsors (overlapping with standalone clients), who collectively contributed $11.7M to AST in support of R&D (or one‑quarter of all R&D revenue).
Over the evaluation period, AST consistently worked with both other government departments (OGDs) and industry, and increasingly with academia and others since 2017‑18, though OGD projects accounted for over half of AST’s revenue.
Other government departments | Industry | Academia and others |
---|---|---|
including 15 federal clients, though most of this work was conducted with DND, Natural Resources (NRCan), and Transport Canada (TC).
|
including 448 SMEs and MNEs
|
including 47 universities, industry associations, municipal transit agencies and provincial government departments.
|
Public / private partnerships to support sustainability
AST also conducted 25 projects with a combination of OGD, industry and other collaborators with funding support from NRCan’s Program for Energy Research and Development (PERD). PERD is a federal, interdepartmental program that funds R&D designed to ensure a sustainable energy future for Canada. It exclusively funds projects that have both a federal partner (e.g., NRC) and an outside organization (e.g., a Canadian business or university).
Relevance
Overall finding: With its recognized expertise and large scale facilities, AST stands out as unique in Canada, although not internationally. AST may strengthen its position if it leverages its unique characteristics and competitive advantage by further integrating its material processing and manufacturing, and vehicle and transportation engineering capabilities. Overall, AST’s research is aligned with the priorities of industry and of the Government of Canada. That said, AST has tended to be reactive to client needs rather than proactively identifying and targeting issues of strategic importance to industry. It has opportunities to more directly contribute to Canada’s competitive advantage, and to consider end‑users from diverse populations in its selection of research activities.
Unique position and opportunity within Canada
Finding
AST is differentiated by its large scale facilities, recognized expertise, and breadth of research. Its inclusion of both materials processing and manufacturing and vehicle and transportation systems engineering under one centre, and its focus on multiple modalities, offers it a competitive advantage.
Sources: client survey, data review, document review, external and internal interviews, peer review
AST’s breadth of research is an under‑exploited competitive advantage
The breadth of AST’s activities from materials/manufacturing to transportation engineering, as well as its coverage of different modalities (e.g., rail, road) was highlighted as a competitive advantage for AST. By contrast, TC’s Motor Vehicle Test Centre and federal agencies in the United States are focused on specific modalities and do not bridge manufacturing and transportation.
This competitive advantage has not been fully leveraged, with AST’s materials and manufacturing and transportation engineering research teams largely working in silos. These teams have worked more with other RCs than one another. Inhibiting factors included limited integration of transportation programs, limited understanding of one another’s competencies, AST’s geographic and linguistic divides, and unique service cultures (e.g., testing versus research).
Integrating research domains
AST has taken recent actions to exploit synergies between sections, including:
- transitioning the VPT program from primarily materials/manufacturing to some transportation engineering projects, joining battery manufacturing and vehicle integration expertise
- requiring program leaders from both sections to identify and submit a joint project proposal for PERD funding
- involving materials/manufacturing teams in planning the new ATSP
- identifying other intra‑AST labour sharing opportunities (e.g., manufacturing staff contributing to vehicle aerodynamics projects)
AST would further benefit from a clearer vision that responds to common challenges in both the manufacturing and transportation sectors. In practice, this would be supported by more strategic project selection.
Unique capacities within Canada
AST’s capacities were found to be unique within Canada though not globally. AST has maintained its relevance with its ability to solve technical problems and conduct strategic R&D for both transportation and manufacturing sector clients.
77% of AST’s clients chose them for its expertise including:
- recognized capabilities in additive and aluminum manufacturing, rail/wheel interaction, and aerodynamics
50% of AST’s clients chose them for its facilities including:
- industrial‑grade presses and production lines
- structural test facilities and environmental chamber accommodate larger scale vehicles
- access to other NRC RCs’ facilities such as aerodynamics and battery safety
Strategic versus reactive project selection
Finding
AST has been more reactive than strategic, particularly in selecting its transportation engineering projects. A shift towards more strategic research that builds on its competitive advantage, would enable AST to have a greater impact for industry and government as opposed to incremental contributions.
Sources: data review, document review, external and internal interviews, peer review
Transportation engineering focused on immediate needs
This section of AST has been largely reactive, serving its clients’ immediate needs (i.e., testing and technical services). It has not proactively identified issues of strategic importance to surface transportation sectors, including grand challenges that cross modalities. This trend stems from a historically technician‑heavy workforce that previously operated under a cost‑recovery model, combined with revenue targets, capital‑intensive infrastructure, limited business support, and legacy relationships (i.e., DND).
AST has acted to reorient this section. Firstly, it has proposed a new program (ATSP) to consolidate existing work and transition to more R&D. It is also strategically reviewing its assets and activities to determine if others can take on service projects (particularly for DND) to free up capacity, hiring more researchers and business developers, developing an industrial R&D group for rail, and exploring R&D opportunities with existing clients (as of 2019‑20).
Materials/manufacturing R&D may focus more on applications for the transportation sector
AST has aligned this section with future directions of the field, as demonstrated by staff having spent far more time on strategic R&D than technical services. It has been successful at identifying collaborative R&D project opportunities with its materials and manufacturing clients, and industrial R&D group members, creating opportunities for co‑authorship and joint presentations.
Per the PRC, this section has more opportunities to link its research (e.g., novel materials and processes) to transportation systems (e.g., transportation networks). The section may also be focused on too many technologies relative to its size, making it difficult to maintain relevance and a leadership status. Focusing more specifically on transportation applications would build on, or be in addition to previous success in technology deployment and commercialization (see Excellence section examples below).
Opportunity to lead on cross‑cutting projects that bridge Canadian transportation engineering and materials/manufacturing
As a national convener, AST can play a leadership role to determine Canada’s vision for navigating technological disruptions in the transportation and manufacturing sectors, especially where they intersect. Building on its competitive advantage, AST could position itself as a made‑in‑Canada solution, and in turn provide Canada with a more competitive and sustainable advantage in both sectors. Though it has begun to integrate its two research domains, the PRC found AST has not done enough to connect disruptions in transportation (e.g., CAV/ITS) and manufacturing (e.g., additives, augmented reality), or those that cross sectors (e.g., digitization, automation) or modalities (e.g., integration of rail and road systems). By selecting projects more strategically and focusing on cross‑cutting issues, AST may better align with Canadian industry, giving it an advantage within international markets. This would also strengthen AST’s alignment with federal economic priorities.
Alignment with industry priorities
Finding
AST’s research is aligned with the current high level priorities of the manufacturing and transportation sectors. It could further support industry by focussing where they intersect.
Sources: client survey, document review, external interviews, peer review
AMP demonstrated relevance
AST has been researching innovative materials and processes that are relevant to Canadian industry (e.g., light‑weighting). It has aligned the AMP with future directions in the field (e.g., additive manufacturing, electrification, greener materials) through industry consultations and its R&D groups.
Proposed transportation program strategically focused
The PRC found the ATSP to have a strong, strategic vision, with a timely focus on CAV/ITS, electrification, and alternative fuels. This will better align AST’s transportation research with industry priorities. Previously, AST’s work in this area was more aligned with the needs of OGDs or client‑specific technical needs. Integrating road and rail modalities will be of particular interest to transit systems looking to put this into practice.
Opportunity to extend its competitive advantage to Canadian industry
AST may better contribute to Canadian competitiveness by further exploiting its own competitive advantage. As noted previously, AST should prioritize work that builds on its relative strengths (i.e., breadth of research, multiple modalities) for the benefit of Canadian firms over foreign firms whenever possible. Towards this, the PRC communicated specific examples for cross‑cutting research. The PRC warned that if AST does not act on this opportunity, leadership in the advanced manufacturing and transportation sectors will continue to shift to other countries. Moreover, while there are opportunities within a rapidly changing transportation sector for independent testing of new technologies, AST’s current model may become less unique as academia is also shifting towards a fee‑for‑service model.
AST activity aligned with industry priorities
Increased battery capacity
- electrification and alternate fuels that focuses on improving batteries, electric motors and fuel cells
Increased exports, reduced import dependence
- work with Canadian automotive OEMs, part suppliers, and the rail sector to improve market competiveness
Procurement of emission free transit vehicles
- electric bus modelling considering battery capacity
Reduced GHG emissions, improved safety
- focus on more energy efficient vehicles with light‑weight materials such as aluminum and advanced composites, as well as enhanced aerodynamics
- focus on zero emission enabling technologies such as low‑cost electric motors and lithium‑based batteries, advanced energy storage technologies and manufacturing of fuel cells for electric vehicles
Reduced operational costs
- rail engineering and vehicle track interaction for predictive maintenance
- cheaper, advanced manufacturing processes (e.g., advanced machine vision for monitoring and control)
Alignment with federal government priorities
Finding
AST is aligned with the overarching NRC objectives and with the priorities of the Government of Canada on the economy, the environment, and national security.
Sources: document review, external interviews, peer review
AST aligned with federal priorities
Economic priorities (i.e., increased sales and export diversification in the advanced manufacturing sector)
- research on additive manufacturing, data engineering and analytics, mechatronics, and robotics
Environmental priorities (i.e., greenhouse gas emission reduction)
- research on alternative fuels, connected and autonomous vehicles, electrification, intelligent transportation systems, and light‑weighting
- work with TC in support of regulation development and implementation, which supports the adoption of technologies for reducing GHGs
- work with TC on guidelines for long trains to reduce derailment risk, and thereby reduce environmental impacts
National security
- provision of testing and technical services to DND related to vehicle performance, instrumentation and hardware, such as communication technologies, for military applications.
Alignment with NRC’s three key objectives
- AST supports innovative businesses for growth through its strategic R&D and technical service work with over 250 Canadian businesses, plus its facilitation of industrial R&D groups and work with the NGen cluster (see Engagement).
- AST advances scientific and technical knowledge through publishing papers and creating intellectual assets, albeit more in material processing and manufacturing (see Excellence).
- AST delivers evidence‑based solutions to inform decisions in government priority areas by working collaboratively with OGDs such as TC and DND.
Other opportunities to support federal priorities
The PRC and external stakeholders identified further opportunities to advance federal priorities. Given its communication technology expertise, AST could play a broader role in supporting other defence priorities such as the cyber‑security of the systems used in Canadian military defense fleets. More broadly, AST could form an industrial R&D group with OGDs and provinces focused on policies that respond to disruptive technologies.
Consideration of end‑users
Finding
There are opportunities for AST to consider end‑users from diverse populations in its selection of research activities, particularly for its work in the area of advanced transportation systems as there are implications for the aging population as well as people with disabilities.
Sources: document review, internal interviews
AST’s programs did not consistently consider diverse populations when selecting projects. Often AST viewed its industry clients as their end‑users, and sought to address their needs, which did not necessarily include consideration of diverse populations. That said, some AST projects, particularly those under its transportation engineering projects, are contributing to social impacts that may improve mobility for persons reliant on public transit systems and in Northern or remote communities (see Social impacts).
Future opportunities for new transportation program to support mobility
AST may support aging populations and persons with disabilities with new technologies in choosing the direction of its new ATSP.
Challenges facing aging populations & persons with disabilities | Potential CAV/ITS solutions |
---|---|
Difficulties in perceiving and analyzing solutions while driving | Intelligent cruise control |
Inaccurate judgement of speed and distance while driving | Automated lane changing |
Mitigating fatigue while driving | Driver condition monitoring system |
Operational inefficiencies of transit systems | Integration of transportation networks (e.g., roads, transit, cycling) |
New facilities for advanced manufacturing to involve diverse populations
More recently, AST has taken preliminary steps in considering how its material processing and manufacturing work may impact diverse populations. In 2019, it committed to considering diverse populations as part of its revised program plan for AMP. Towards this, it is in the process of opening a new advanced manufacturing facility in Winnipeg, Manitoba, where it is partnering with both University of Manitoba and Red River College. These institutions have large Indigenous student bodies which will bring new perspectives to AMP.
Why is considering diverse populations important? The Government of Canada’s priority is to consider GBA+ in all of its activities. GBA+ assess how diverse people (e.g., gender, race, ethnicity, religion, age, physical differences etc.) may experience policies, programs, and initiatives.
Engagement
Overall finding: AST’s visibility among and engagement with stakeholders could be greater, particularly in the research community. On one hand, AST’s work in material processing and manufacturing has involved a broad and diversified range of players throughout the manufacturing value chain. This has been facilitated significantly by its industrial R&D groups, as well as industry consultations and working with the NGen cluster. On the other hand, AST’s work in vehicle and transportation systems engineering was primarily to support OGDs, with relatively low engagement with academia and Canadian industry. These gaps must be addressed in order to effectively implement its proposed ATSP.
Engagement strategies and awareness
Finding
Awareness of AST has room to grow. It has employed a variety of strategies, including a unique R&D group model that could be implemented across AST programs in order to increase engagement.
Sources: bibliometric study, client survey, document review, external and internal interviews, peer review
Figure 3 ‑ Text version
Long description of The level of awareness of AST, per its clients
The figure is a bubble chart that depicts the perceived level of awareness of AST across relevant research sectors, according to AST’s clients. There are three bubbles:
- First (left‑most) bubble is the smallest, and represents the 20% of AST clients who agree that most sector stakeholders are aware of the Research Centre
- The middle bubble is the largest, and represents the 50% of AST clients who agree that some sector stakeholders are aware of the Research Centre
- The last (right‑most) bubble is medium‑sized, and represents the 31% of AST clients who agree that few sector stakeholders
Note: Percentages do not add up to 100% due to rounding
AST’s outreach efforts have been somewhat successful. Strategies have included one‑on‑one outreach, participation in events, conferences, and local and regional initiatives, relationship development with SMEs through NRC‑IRAP, publishing, and founding and leading industrial R&D groups (see next slide). Awareness of AST in the manufacturing sector may have been inhibited somewhat by its branding (e.g., exclusion of ‘manufacturing’ from its name).
Staff from both AST sections participate in national and international committees. Materials/manufacturing staff tended to be on academic committees (e.g., Thermal Spray Society) while transportation staff tended to be on Canadian and US federal committees (e.g., US Dept. of Transportation), although this was not tracked systematically.
New opportunities to engage research community and industry
More participation in the research community, such as greater publications and attendance at events/conferences, could raise awareness. Opportunities to attend conferences/trade shows should be extended to junior and technical staff to keep them abreast of emerging issues, and to add more total connections between AST and academia. The PRC also highlighted that publishing and revenues should not be seen as competing priorities, as the former may attract new clients.
Other engagement strategies highlighted by surveyed client as the most effective included increased outreach by executives, advertising in trade publications, and more NRC webinars.
Industrial R&D groups
Finding
AST’s industrial R&D groups have proven effective at fostering collaboration. They are a unique innovation that AST has successfully spearheaded at the NRC.
Sources: data review, document review, external and internal interviews, peer review
R&D groups an effective model to engage industry
The NRC facilitates collaborative, multi‑client industrial groups that allow members to share R&D costs and risks. Members pay dues and collectively identify and deliver R&D projects, while other public and industry stakeholders provide sponsorship. This model is an effective means to expand and sustain engagement, and ensure research priorities are aligned with industry trends.
AST has eight of NRC’s nine R&D groups, mostly associated with materials and manufacturing subsectors. Collectively, members and sponsors provided $11.7M in revenues during the evaluation period, accounting for over one‑fifth of the manufacturing section’s total revenues. Members received access to:
- a network of peers in a non‑competitive environment
- collaborative projects at an affordable cost
- the NRC’s R&D experts, facilities, and technological capabilities
- technologies, intellectual property (IP) and R&D results (see below for more on IP access for members)
Groups engaged 100+ materials/manufacturing businesses
Industrial R&D groups have expanded AST’s reach throughout the manufacturing value chain, including a large number of businesses that had not otherwise worked with AST (two‑thirds of members). Groups active during the evaluation period include:
- ALTec (Aluminum manufacturing technologies)
- CSAM (Cold spray additive manufacturing)
- LiBTec (Lithium battery technology)
- SIGBlow (Blow molding modelling and simulation)
- SNAP Composites (Short, novel, affordable processes for composite manufacturing)
- STAMP Composites (Stamping technology for automotive manufacturing and processing of composites)
- Surftec (Advanced surface engineering technologies for harsh environment applications)
Most group members were Canadian businesses (except for SIGBlow). Sponsors included NRCan, as well as provincial agencies and industry associations. The latter two are mainly from Quebec where industrial research consortia provide strategic and financial support to their respective sectors (their support to the groups was purely financial).
New groups to build on transportation engineering expertise
The transportation engineering section launched its first R&D group in 2019‑20 with a focus on the rail sector (see next slide). The R&D group model may be used to address engagement gaps in this sector. AST plans to launch at least one new group over the next five years. The PRC suggested new groups to build on AST’s strengths such as: a regulatory / policy group with OGDs and provinces, a CAV group with policy and industry players; and, heavy and light rail transit groups with operators.
Transportation engineering stakeholders
Finding
AST’s work in vehicle and transportation systems engineering has largely served a few key accounts, and engagement with academia and Canadian industry has been low. As it ramps up CAV/ITS research, AST will need to broaden their client base to include key players in these spheres.
Sources: data review, external and internal interviews, peer review
Most work with two federal departments, one foreign business
AST’s transportation engineering projects have mostly served three clients, DND (45% of revenues), TC (22%), and an American railcar manufacturer (8%). Remaining revenues came from projects with 123 other clients and collaborators. In industry, it worked with an equal number of Canadian and foreign firms, but did more work for the latter. Most work with other organizations was with Canadian and American transit agencies.
Rail group attracting new collaborators
AST launched an industrial R&D group regarding wheel‑shelling called RailWheelS in 2019‑20. The PRC said it was promising at this early stage as AST had attracted a mix of both Canadian and foreign industry involvement but it was concerned that AST was missing Canadian Class 1 operators such as CN and CP.
AST is also a sponsoring partner of the International Collaborative Research Initiative (ICRI) on Rolling Contact Fatigue and Wear of Rails and Wheels, an informal group of industry, academia, and OGDs. The rail team leads the initiative from a management perspective and also provides technical leadership and support to the international collaborative R&D projects.
Low engagement with Canadian industry and academia
Compared to materials/manufacturing, AST has been relatively less successful expanding its reach in the transportation sector. Engagement was inhibited by limited business development support over the evaluation time period (e.g., insufficient resources and turnover among client relationship leaders), although it now has dedicated business development support for this area. Transportation programs and service lines each had unique areas where engagement was low or had room for improvement:
- Trucking and heavy duty vehicles: for‑hire fleet operators; only indirectly via industry associations (e.g., Canadian Transportation Equipment Association).
- Rail: rail car manufacturers and other firms in rail R&D.
- Defence: industry overall, but particularly MNEs.
AST has engaged relatively little with universities in transportation engineering, such as major players in CAV/ITS (e.g., McMaster, University of Ontario Institute of Technology, Waterloo). The PRC recommended that partnerships with academia would help AST supplement capacity and facilitate a shift towards the more fundamental, intermodal research it plans to pursue under the ATSP. It would also increase its market intelligence, ensuring they initiate unique research and distinguish themselves.
Materials / manufacturing stakeholders
Finding
AST’s work in material processing and manufacturing has engaged an appropriate mix of clients and collaborators (e.g. small and large businesses, Canadian and foreign, OGDs, and universities).
Sources: data review, document review, external and internal interviews, peer review
Broad engagement, no significant gaps
AST had 290 unique clients and collaborators for its material processing and manufacturing projects. It worked primarily with Canadian industry (45% of revenues), followed by foreign industry (27%) and OGDs (26%). It also worked with universities and industry associations on multi‑partner projects (i.e., with a Canadian business).
Co‑authorship with academia, industry
In addition to projects, material processing and manufacturing staff co‑authored publications with external partners. AST’s collaboration rate with industry (20% of publications) was twice the NRC average (11%). Most academic collaborators were based in Quebec, a trend driven by location and third‑party funding requirements (i.e., industrial research consortia in Quebec that mandate collaboration with institutions in that province).
New opportunities to engage
AST is working with the NGen cluster, a non‑profit industry‑led consortia that matches federal funding to businesses and other stakeholders, based in southwestern Ontario. There is potential for NGen to connect AST with more Canadian SMEs, particularly through AST‑London, but this has yet to be realized (e.g., five collaborative projects were identified but unapproved as of 2018‑19).
Resources
Overall finding: AST has had sufficient resources to meet client needs to date, and has been building and transitioning its capabilities for R&D in new and emerging areas. However, AST’s ambitions outweigh their capacity and budget forecast. At this critical point, AST will need to identify and implement new strategies to address talent and facility gaps in order to meet emerging client needs and to position itself as a research leader.
Demonstrated agility and strategic growth
Finding
Over the evaluation period, AST’s capacity met client needs. It has began to build and transition its capacities from technical services to strategic R&D, in order to enter new and emerging areas.
Sources: client survey, data review, external and internal interviews, peer review
Meeting client needs
AST has had sufficient capacity to fulfill its existing lines of work. Most clients surveyed (70%) felt NRC has had the necessary expertise to fulfill project objectives. The PRC was impressed by AST’s agility in responding to a wide range of technology requests.
Accessing broader NRC expertise
AST has built and maintained its capacity by drawing on outside labour. Overall, one quarter (26%) of labour for its projects came from other RCs, a trend consistent over the evaluation period. In particular, AST:
- relied on EME and its expertise in energy storage for half of the labour needed to implement VPT projects
- drew from AERO for a mix of programs, but most notably for FF2020
- has increasingly drawn on DT as projects require more digitization expertise
This poses a risk for AST. To date, staff reported some challenges in delineating research areas and accounting for revenue attribution. Also, AST may find itself competing for these resources as NRC launches new, multidisciplinary programs that cross RCs.
Figure 4 ‑ Text version
Long description of Hiring more researchers to increase R&D
The figure is a vertical clustered bar chart that depicts the number of researchers, technicians and students, respectively, employed at AST each year throughout the evaluation period. While the number of technicians declines slowly over time, there is a notable upward trend in the number of researchers employed, reflecting AST’s efforts to increase R&D. The number of students also increases drastically in 2018‑19, from 5 to 19.
Researchers | Technicians | Students | |
---|---|---|---|
2013‑14 | 101 | 107 | 1 |
2014‑15 | 110 | 99 | 6 |
2015‑16 | 116 | 96 | 5 |
2016‑17 | 133 | 97 | 6 |
2017‑18 | 141 | 93 | 5 |
2018‑19 | 141 | 92 | 19 |
Note: This figure excludes management and administration.
New researchers, students in emerging fields in materials/manufacturing
AST grew this section from 120 to 159 staff. After an NRC‑wide cap on budgets was imposed in 2018‑19, it continued building capacity by taking on more students. The directorate in AST‑London has been budgeted to hire new staff to grow its manufacturing and CAV labs, thus supporting both sections of AST.
Transitioning the TEC towards R&D
AST has been strategically replacing technicians that have left the NRC with researchers, while maintaining relatively stable staff numbers (up from 89 to 93). In 2013‑14, 38% of the TEC staff in Ottawa were researchers; by 2018‑19, their share increased to 49%.
Gaps in expertise and capacity pressures
Finding
AST lacks the competencies needed to exploit its innovations in material processing and manufacturing, or to enact the vision of the ATSP. Existing staff have had insufficient opportunities to develop new competencies due to competing revenue priorities.
Sources: client survey, data review, external and internal interviews, peer review
Missing expertise and critical mass to respond to emerging demands
Staff reported that they had managed to meet clients needs over the evaluation period but are ill‑positioned to respond to new or more complex demands. Moreover, almost half of clients (48%) said AST had some capacity gaps. Identified gaps included digital fields such as artificial intelligence (AI), CAV/ITS, cyber‑security, and modeling and simulation. In some of these areas (e.g., AI, modeling and simulation) AST has expert staff but lacks critical mass. Gaps have been mitigated in part through strategic hiring, accessing labour from other NRC RCs, and internal competency development.
Materials/manufacturing staff given more development opportunities
AST has developed and broadened staff competencies through exploratory R&D projects but has had difficulty balancing this with revenue generation. Though staff have been afforded more time for these internal projects (after a major decrease in 2015‑16; see right), transportation engineering staff, especially technicians, have had limited opportunities to develop new competencies. This may undermine the transportation engineering section’s ability to respond to emerging industry needs and deliver the ATSP in the short‑term (i.e., before it can hire new researchers). By contrast, materials/manufacturing staff have been afforded more development opportunities, in areas such as machine learning and manufacturing digitalization, supporting their ability to conduct R&D that addresses emerging client needs.
Figure 5 ‑ Text version
Long description of Limited opportunities for transportation engineering staff to develop new competencies.
The figure is a horizontal line chart that compares the percentage share of working time spent each year by both researchers and technicians within the Materials/Manufacturing and Transportation Engineering research areas, respectively, contributing to internal R&D projects. The data shows that researchers and technicians within the Materials/Manufacturing research area spend a considerably greater percentage of their time contributing to internal R&D projects than do researchers and technicians within Transportation Engineering, with technicians in this area contributing the lowest share of working time. This trend suggests that there are limited opportunities for Transportation Engineering staff, especially technicians, to develop new competencies.
Fiscal Year | Materials / Manufacturing Researchers | Materials / Manufacturing Technicians | Transportation Engineering Researchers | Transportation Engineering Technicians |
---|---|---|---|---|
2014‑15 | 15% | 21% | 9% | 1% |
2015‑16 | 10% | 7% | 2% | 1% |
2016‑17 | 8% | 10% | 6% | 1% |
2017‑18 | 14% | 14% | 6% | 0% |
2018‑19 | 16% | 14% | 5% | 1% |
Note: Detailed timesheet data was not available for 2013‑14.
Demographics and education
Finding
The materials/manufacturing and transportation engineering teams have very different educational backgrounds, which align with their distinct cultures (i.e., R&D focus versus cost recovery). AST has begun to expand its capabilities by hiring more students and strategically replacing departing staff, which also provides an opportunity to improve employment diversity and equity.
Sources: data review, internal interviews, peer review
Figure 6 ‑ Text version
Long description of Differences in AST sections’ educational profiles
The figure is a 100% stacked bar chart that compares the educational profiles of employees within AST’s Transportation Engineering and Materials/Manufacturing research areas. The data is limited to research and technical staff (excludes management and administration) and is valid as of March 31, 2019. The data shows that a majority of Transportation Engineering staff hold either a Bachelor’s degree or certificate/diploma, while a majority of Materials/Manufacturing staff hold either a Doctorate or Master’s degree.
Research Area | Doctorate | Master's | Bachelor's | Certificate / Diploma | High school |
---|---|---|---|---|---|
Transportation Engineering | 8% | 12% | 31% | 36% | 13% |
Materials/ Manufacturing | 45% | 17% | 13% | 25% | 0% |
Note: Data limited to researchers and technicians as of 2018‑19, excluding management and administration, and those who did not have education on file (10% of total scientific and technical staff).
Hiring next generation of young and diverse researchers
AST met its 2018‑19 equity targets for women (i.e., 22%). Of AST management, 29% were women. Of researchers and technicians, 15% were women. Of administration, 95% were women.
Gender split by section, with 19% of manufacturing researchers and technicians versus 7% of transportation engineering.
Other equity groups were underrepresented at AST, based on self‑identifications volunteered by staff versus workforce availability per Statistics Canada. Visible minorities made up 14% of staff, below the 22% target. Targets for Aboriginal people (2%), and persons with disabilities (4%) were also not achieved (actual figures suppressed to maintain individual privacy).
Hiring more students may improve equity in the long‑run as many co‑op students find permanent positions with AST. In 2018‑19, 42% of students were women. Partnerships with schools with large Indigenous populations are also being developed (i.e., University of Manitoba, Red River College).
Materials / manufacturing teams
These teams are scientist‑heavy, and tend to be very focused in specific areas and lower TRL research. The PRC suggested AST could supplement these teams with new staff with more business acumen and sufficient technical understanding (e.g., Master’s level graduates), in order to support greater commercialization of AST’s innovations in materials and processes
Transportation engineering teams
These teams are lacking graduate‑level researchers, and tend to be focused on technical services. Attracting topic‑diverse researchers would enable AST to respond to changing directions and exploit new opportunities. Hiring recognized leaders may attract more interest from industry and other sector players.
Retire‑and‑replace
Many employees are mid‑to‑late career. By 2024, 28% of AST’s staff will be eligible to retire, and nearly half (44%) by 2029. This aligns with the NRC overall. AST’s staffing strategy has, in part, relied on ongoing turnover to transition its workforce towards more R&D. However, fewer staff have retired than anticipated, slowing this process and contributing to budget pressures.
Equipment and facilities
Finding
AST’s facilities met the needs of most clients, despite lacking planned investments. Equipment and facility gaps may inhibit AST’s ability to address emerging needs and position itself as a leader.
Sources: client survey, data review, internal and external interviews, peer review
Investment lacking, but clients satisfied
Most clients surveyed (61%) felt the NRC has the facilities to fulfill their objectives, though 37% said there were some gaps. External stakeholders described the facilities as appropriate, with some calling them ‘state‑of‑the‑art.’ The PRC described the facilities as being of a good calibre and sufficient for a range of different industries, technologies, and vehicles. However they observed that AST had invested in current facilities rather than moving on and into new facilities to support future research priorities. To position itself among global leaders in China, Europe and the US, AST needs to make major investments, specifically to maintain relevance in rail.
Throughout most of the evaluation period, AST was unable to consistently secure the major capital it needed in order to invest in and upgrade its facilities, instead relying on interim investments and third‑party facilities to meet its needs. In 2018‑19, however, it did secure major capital to build a new battery manufacturing facility in Boucherville for its VPT program.
Materials/manufacturing needs new equipment, exploring partnerships
Staff say it is difficult for them to innovate with their current facilities. They have dedicated time to adapting existing equipment to meet client needs but still have had to turn away some new business opportunities. Specific gaps include:
- digital manufacturing equipment to create a demonstration facility of a digital factory at AST‑London
- HPC and IT software, namely computer clusters and digital simulation software
- replacements for its laser system for machining and scanning electron microscope
AST has relied on academia for specialized equipment. It is exploring arrangements to access or upgrade facilities by partnering with universities to leverage third‑party funds.
Scope of transportation engineering work limited by existing facilities
Staff reported some facilities are out‑of‑date and missing features, preventing them from competing with others and possibly diminishing client confidence. Clients interviewed were mostly satisfied with facilities but noted they cannot do some large‑scale tests. Specific gaps include:
- climatic chamber requires regular repairs, lacks solar simulation
- wind tunnels too small fit a full‑length tractor trailer
- facilities for freight railcars too small to conduct a fatigue test for all load conditions
AST has relied on TC’s Motor Vehicle Testing Centre for on‑road testing to cover some facility gaps.
Budget limitations
Finding
AST is at a critical point as it is required to reduce its spending at the same time intends to enter new research areas. AST will need to strategically invest in new resources and facilities, and leverage partnerships, that build upon its competitive advantage (i.e., breadth of research, multiple modalities) rather than continue business‑as‑usual.
Sources: data review, document review, external and internal interviews, peer review
Increasing salaries poses budget challenge for AST
AST management expressed concern that ongoing expenses may constrain its ability to build and transition its R&D capabilities, a concern echoed by the PRC. Salaries, which have increased over recent years, were identified as a key driver of rising costs that restrict AST’s ability to invest in new capabilities. Over the same time, AST has spent less than it had planned on minor capital investments (see table).
Causes for AST’s increasing salary costs lay in part outside of AST’s control. NRC’s budget reallocation in 2017‑18 capped operating budgets. Salaries still grew as anticipated retirements of more highly paid staff did not occur (e.g., in 2018‑19, 11% were eligible to retire but only 2% did). Moreover, base salary rose over the last two years due to the ongoing climb, by employees, in their respective salary scale.
Managing its increasing costs will be a challenge for AST as the RC is expected to reduce its costs in the short‑term. As part of the NRC 2017‑18 budget reallocation, all RCs were required to reduce spending by 10% over the next five years. AST does have HR and facility strategies in place to guide this process, but additional strategies may be necessary in order for it to deliver on its new program in light of budget limitations.
FY14 | FY15 | FY16 | FY17 | FY18 | FY19 | |
---|---|---|---|---|---|---|
Total expenses | $45.3M | $43.9M | $47.2M | $51.7M | $56.8M | $55.2M |
Salaries | $26.5M | $25.2M | $26.8M | $27.6M | $28.5M | $31.0M* |
% of total | 59% | 57% | 57% | 53% | 50% | 56% |
Minor capital investments | $3.9M | $3.1M | $3.3M | $4.0M | $2.0M | $2.2M |
% of total | 9% | 7% | 7% | 8% | 4% | 4% |
*Total expenses in 2018‑19 were within budget but salaries were 2% over budget. |
Strategic investments, partnerships necessary for success
Capital investments are necessary if AST wishes to position itself as a leader. AST needs a vision and plan to better integrate its two sections and prioritize resource allocation accordingly. Beyond operationalizing existing HR and facility strategies, the PRC stressed that AST should pursue new strategies, such as investing in resources that support both research domains, partnering with more third‑parties to access both competencies and facilities, and, where possible, by using more modeling and simulation to reduce resource‑intensive physical testing.
Excellence
Overall finding: AST has demonstrated scientific excellence in material processing and manufacturing, and transportation engineering in specific niche areas. This excellence has not been demonstrated through scientific publications, but rather through the development and optimization of technologies and process with the potential to advance the automotive and surface transportation sectors. AST will need to address resource, focus, and engagement issues in order to become a leader in advanced ground transportation systems.
Expertise, innovation and technology deployment
Finding
AST is recognized for its expertise in both materials/manufacturing and transportation engineering in specific niche areas. It has optimized and developed new technologies and processes with the potential to advance both sectors.
Sources: document review, external and internal interviews, peer review
Material processing and manufacturing
AST has demonstrated scientific excellence in areas such additive manufacturing, batteries, biomaterials, cold spray, light‑weighting, and software development for moulding. The PRC also recognized AST’s significant promise in stamping and forming of printable electronics, a relatively new area for AST. Specific contributions to the fields includes:
New fabrication method for electric motors, namely cold spray technology, to fabricate magnetic materials such as permanent magnet components. This has the potential to improve performance and reduce costs, as demonstrated by a Canadian SME that deployed new components designed with this technology.
Developed bio‑alternative to petroleum‑based materials using lignin pellets and lignin‑polymer materials recycled from pulp and paper manufacturing by‑products. This is a natural alternative to petroleum‑based thermoplastics with a short time to market. A trial with agricultural films is underway.
Vehicle and transportation systems engineering
AST’s rail research is extensive and the team has a historically strong reputation. Its work on instrumented wheelsets is particularly noteworthy. It was also recognized for its innovative test methods for aerodynamics of rail and heavy duty vehicles. Specific contributions to the field include:
Municipal transit authority in a major American city deployed AST autonomous instrumented wheelset technology on its subway line to monitor vehicle/track health and identify derailment potential. The client also described AST’s work on rolling rail contact fatigue as ahead of Transportation Technology Center, Inc., who uses the NRC’s instrumented wheelset technology for evaluating new railcars design.
New testing protocol for electric vehicle safety, developed by AST, was adapted by an international OEM into its design of future electric vehicle battery packs.
Intellectual property
Finding
AST has contributed to the advancement of knowledge through its inventions and new IP. Its sharing of IP through the ALTec R&D group is an innovative practice to support members and demonstrate leadership.
Sources: data review, internal interviews, peer review
AST inventing, disclosing IP for material processing and manufacturing
As of 2018‑19, AST was one of NRC’s most prolific generators of new IP. Throughout the evaluation period, AST disclosed 85 new inventions or other IP (see figure), nearly all of which related to materials/manufacturing. It also filed patent applications for 17 unique inventions, all but one of which related to materials/manufacturing. As of summer 2019, three of these applications resulted in an issued patent.
On average, AST disclosed 10 new inventions each year. The plurality was related to metal products manufacturing (4 per year), followed by polymer and composites products (3), and vehicle propulsion technologies (2). While most of these inventions had applications for material/manufacturing, three VPT inventions had application for vehicle and transportation systems engineering.
AST spearheading best practice for knowledge distribution to industry
AST has generated and shared IP through its industrial R&D group model. In 2018‑19, the ATC in Saguenay catalogued 22 knowledge products, including algorithms, methodologies, and simulation tools, for exclusive use by members of the ALTec industrial R&D group. This accumulated ‘know‑how’ represented five years of internal R&D conducted for the group. While AST and other RCs have generated and disclosed similar types of IP (e.g., methods), none have done so in this manner, with AST effectively creating an exclusive knowledge catalogue for innovative businesses. The PRC highlighted this as a best practice that AST’s other industrial R&D groups should emulate.
Figure 7 ‑ Text version
Long description of AST consistently developing new innovations
The figure is a vertical clustered bar chart that depicts the number of new inventions vs. the number of new copyrights, marks and/or trade secrets disclosed by AST each year throughout the evaluation period. While outputs are lower in fiscal years 2015‑16 and 2016‑17, the data shows that AST consistently developed new innovations throughout the evaluation period. In addition to the new inventions, copyrights, marks and trade secrets produced in 2018‑19, AST also disclosed 22 new IP, exclusive to ALTec industrial R&D group members.
Fiscal Year | Inventions | Copyrights, marks or trade secrets | Exclusive IP for ALTec industrial R&D group members |
---|---|---|---|
2013‑14 | 9 | 0 | 0 |
2014‑15 | 11 | 0 | 0 |
2015‑16 | 4 | 1 | 0 |
2016‑17 | 8 | 2 | 0 |
2017‑18 | 14 | 1 | 0 |
2018‑19 | 12 | 1 | 22 |
Note: This is limited to reported IP and would exclude any inventions AST has chosen not to disclose at this time.
Scientific influence
Finding
AST has deployed new technology and IP that has had a positive impact on clients, but the scientific impact of its publications has been lower than its comparators.
Sources: bibliometric study, peer review
Most publications in material processing and manufacturing
Materials/manufacturing staff produced 327 publications versus only 27 from transportation engineering, between 2013 and 2018. Output by the former however declined after 2013 due to an NRC‑wide shift towards technical services and increasing revenue targets (i.e., 100 materials/manufacturing publications in 2013 versus 36 in 2018). The transportation engineering section has improved somewhat as they were already focused on cost‑recovery activity (no publications in 2013), but have begun reorienting towards R&D (9 publications in 2018). Per the PRC, publishing should be part of its engagement and business growth strategy, rather than in competition with it, as publications boost their authors’ reputation and profile.
Relatively low impact within academia
AST publications were not well cited relative to NRC nor Canada overall. While Canada and the NRC both averaged FWCI scores of 1.5 for the six‑year evaluation period, AST averaged only 0.9. Both sections lagged behind most comparator (see figure). Additionally, its publications were also less cited in patents (14 per 1,000 publications) relative to NRC (48) or Canada (25).
Opportunity to establish industry‑relevant outputs and targets
As AST’s target audience is industry, scientific excellence targets should be expanded beyond publication metrics to include indicators that capture its influence on innovative businesses (e.g., conference attendance, trade show presentations, and technology demonstration and deployment). This would be especially relevant for its transportation engineering section, which has been focused on technical services over publication.
Figure 8 ‑ Text version
Long description of Impact lags comparators
The figure is a horizontal bar chart that compares the Field‑Weighted Citation Index (FWCI) of AST’s Materials/Manufacturing and Transportation Engineering research areas, respectively, to similar organizations in their sector. AST’s FWCI was determined through a bibliometric study that was conducted by calendar year (not fiscal year) and that covered all publications from 2013 through 2018. In both cases, AST’s FWCI is lower than that of most comparators, suggesting that AST’s publications have had a lesser impact compared to other peer reviewed publications across the relevant research domains.
Organization | FWCI |
---|---|
NRC‑AST Transportation Engineering | 0.44 |
Transportation Technology Center Inc. | 0.76 |
National Renewable Energy Lab | 1.84 |
Fraunhofer Institutes Transp. & Infrastructure Systems | 2.08 |
Organization | FWCI |
---|---|
NRC‑AST Material Processing and Manufacturing | 0.92 |
Fraunhofer Institutes Additive Mfg Technologies | 0.36 |
Fraunhofer Institutes Mfg Tech. &Adv. Materials | 1.32 |
Fraunhofer Institutes Process Eng. &Packaging | 1.56 |
Fraunhofer Institutes Mfg Eng. &Automation | 2.19 |
Fraunhofer Institutes Laser Technology | 2.32 |
CSIRO Mfg Flagship | 2.63 |
Notes: Bibliometric analysis was conducted by calendar year, not fiscal year, and covered all publications from 2013 through 2018. FI are Fraunhofer Institutes.
What is the Field‑Weighted Citation Index? The FWCI is a normalized indicator to gauge relative performance of a publication in terms of citations. It takes into consideration the number of publications and publication norms in different research domains.
Position for future leadership
Finding
AST’s potential to lead in advanced ground transportation systems is inhibited by resource gaps, limited examination of intermodal systems or cross‑cutting problems to date, and isolation from key leaders in the field. Attaining leadership will demand both internal and external collaboration.
Sources: bibliometric study, data review, document review, external and internal interviews, peer review
Promising, relevant research direction
The proposed ATSP will focus on four research domains, namely rail, automotive and specialized mobility (e.g., trucks and heavy duty vehicles), CAV/ITS, and electrification and alternate fuels. The PRC and external stakeholders concurred that the ATSP has a promising, relevant research direction overall (e.g., connectivity, automation, sustainability) and that its planned niches in perception and mapping systems, and adapting technologies to the Canadian context (e.g., signage, conditions), are appropriate.
AST’s research activity to‑date has helped position it to be a leader in advanced transportation systems. Its work in rail, in some areas, is leading‑edge within North America (e.g., instrumented wheelset). AST is also well‑positioned in research areas such as CAV, for which is it hiring, and electrification, for which it is building new facilities.
Nascent capacities in competitive fields
Although it is well‑positioned and has a promising direction, AST lacks the resources (e.g., competencies, critical mass, facilities) to fulfill the ATSP’s objectives. For instance, the emphases on CAV/ITS and electrification—albeit appropriate—are nascent and more resources will be necessary. This may be mitigated in part through collaboration with academia, but will also require investments in experimental facilities.
Limited examination of intermodal systems, cross‑cutting problems
There has been little synergy between the current transportation programs (i.e., FF2020, RVTO, and VPT). AST’s transportation engineering teams tended to work in mode‑specific silos rather than collaborate. The PRC identified synergistic research opportunities such as rail and road systems integration, common vehicle dynamics problems related to curving, and infrastructure layout of transportation networks.
Isolated from key leaders in the field
AST has had insufficient collaboration with academia in the area of vehicle and transportation systems engineering. This is essential to engage in cutting edge research and develop expertise. This should be seen as an opportunity to collaborate rather than compete. Potential collaborators identified by the PRC include:
- Waterloo Centre for Automotive Research at the University of Waterloo
- Institute for Automotive Research and Technology at McMaster University
- Electronic Vehicle Research Centre at the University of Toronto
To facilitate collaborations with universities, AST could set up master research agreements between the universities, industry and AST to define how IP is handled, technology is transferred and how joint publications can be produced.
Impact
Overall finding: Through its work in both material processing and manufacturing, and vehicle and transportation systems engineering, AST has contributed to the economic growth and prosperity of the Canadian manufacturing and transportation industries, supported government operations, policy, and regulations, and contributed to longer term environmental impacts. Additionally, its work in transportation engineering is contributing to a safer, more secure, more efficient, and more reliable Canadian transportation system, and its work in manufacturing is positioned to create new, better career opportunities to Canadians in advanced manufacturing. That said, AST has opportunities to better track its achievements.
Economic impact
Finding
AST positively impacted its industry clients, who reported adopting new technologies, releasing new or improved products, and increasing their competitiveness. Its industrial R&D groups provided members increased innovation capacity and opportunities to strike new partnerships in the manufacturing sector.
Sources: client survey, data review, external interviews, industrial r&d group member surveys
Different impacts between transportation and manufacturing clients
Transportation engineering clients were more likely to report increased competitiveness, access to new markets, and reduced product‑time‑to‑market. For example, AST provided a Canadian rail operator with evidence of their routes’ superior ride quality, supporting their case to have dedicated tracks. AST also helped industry meet regulations, for instance working with the Canadian Transportation Equipment Association to develop rear impact guard designs that meet TC regulations.
Materials/manufacturing clients were more likely to report increased productivity, and new partnerships. For example, work on light‑weighting for flight decks of SUVs and minivans led to a new value chain including multiple Canadian firms.
R&D groups increased innovation capacity, partnership opportunities
R&D groups gave members opportunities to collaborate, and stay abreast of the latest developments in their fields. Member surveys found that participants increased innovation capacity (83% of ALTec members) and led to new to business connections (88% of Surftec members). Interviewees welcomed access to unique and useful technologies (e.g., SIGBlow software, ALTec IP catalogue).
Opportunity to conduct follow‑up on and track economic impact
If AST were to track foreign investment, revenue growth, and employment growth, among its industry clients and R&D group members, it would be able to better understand how and where it is having the greatest economic impact for Canada.
Figure 9 ‑ Text version
Long description of Positive impacts for industry
The figure is a horizontal clustered bar chart that presents the percentage of AST clients who reported experiencing one or more positive impacts attributable to their work with AST. There are 7 impact measures listed, and the client response data is broken down by Materials/Manufacturing and Transportation Engineering research sections:
Impact measure | Materials/Manufacturing Clients | Transportation Engineering Clients |
---|---|---|
Reduced product‑time‑to‑market | 29% | 40% |
Improved overall productivity | 38% | 28% |
Increased partnerships | 44% | 32% |
Access to new markets | 41% | 56% |
Increased competitiveness | 47% | 72% |
New or improved products or solutions | 62% | 70% |
Adoption of new technologies | 65% | 78% |
“AST put together a fantastic consortium of end‑users such as ourselves. We were able to take that information and spin it off into a program to further advance the technology for our applications.”
Support to government
Finding
AST has contributed to government policy and regulations, particularly through its transportation engineering and VPT programs, as well as supported government operations.
Sources: client survey, external interviews
Contributing to regulations while supporting economic growth
- contributed to hydrogen safety and measurements standards and codes with TC and Environment and Climate Change Canada (ECCC)
- developed heavy and light duty vehicle GHG emission criteria for North America with TC and ECCC
- developed heavy duty vehicles weights and dimensions regulations for harmonizing standards among provinces and territories in an effort to reduce trade barriers
- provided recommendations to TC on the effectiveness of automated train brake test technology
- supported TC in its involvement on an international working group developing standards and regulations for electric vehicles
Operational changes and cost savings
- for DND, improved the efficiency and preparedness of operational fleets, such as light armored vehicles (see picture)
- contributed to some cost savings and operational changes at ECCC and TC
Opportunity to regularly track and document policy influence
If AST tracked how it has impacted policy (e.g., changes to codes), it would be able to better understand and communicate how it supports federal priorities.
Social impacts
Finding
AST has contributed to a safe, secure and reliable transportation system, particularly for the benefit of transit‑dependent and remote communities. It has created new opportunities in advanced manufacturing for students, and is positioned to support that sector more broadly with NGen.
Sources: client survey, data review, document review, external interviews
Making it safer to travel Canada by road or rail
AST has contributed to social impacts, primarily through its transportation engineering and VPT programs. Overall, over two‑fifths of AST’s clients said their project had (16%) or would (27%) contribute to a safer, more secure and reliable transportation system in Canada. Socially impactful projects included:
- determining the cause of failed rail (e.g., projects with a Canadian regional railway)
- ensuring safety and increasing operational efficiency and longevity of public transit and passenger rail systems (e.g., projects for municipal rail operators in major Canadian cities), which supports populations more dependent on transit (e.g., older Canadians, persons with disabilities).
- supporting adaption to climate change / climatic conditions, particularly for Northern and remote communities (e.g., projects with VIA Rail/Infrastructure Canada on conditions in North)
- facilitating adoption of new trucking innovations (e.g., boat tails, rear impact guards) to improve vehicle safety for occupants and traffic participants, and to improve fleet efficiency.
New career opportunities in advanced manufacturing
AST has aligned the AMP with the Economic Strategy Table for the Advanced Manufacturing Sector; which itself seeks to address skilled labour shortages. Towards this, AST has hired more advanced manufacturing students from its university partners in Ontario and Quebec, tripling its student count from 4 to 15 in 2018‑19 (as well as increasing transportation students from 1 to 4). AMP was also aligned with the NGen cluster. AST supported NGen’s development (e.g., program design, project selection) and will work directly with some of its clients, actions which will support growth in this sector.
“AST stimulates the industry by offering high‑profile job opportunities or internships in areas that are up‑and‑coming. It helps us to continue to grow and keep our workforce on this side of the border.”
Environmental impacts
Finding
AST’s work in both material processing and manufacturing, and vehicle and transportation systems engineering has contributed to environmental benefits.
Sources: client survey, document review, external interviews
Half of AST clients working toward a cleaner environment
AST’s activities in both its research areas are united by an overarching aim at generating environmental benefits. Half of AST’s clients (51%) said their work with AST had contributed to a cleaner environment or was expected to do so in the future (16% has contributed; 35% in the future). In both of its research areas, many of AST’s strategic research and technical service projects had ‘green’ goals such as reducing fuel consumption, developing eco‑friendly products, and reducing environmental risks such as train derailments or nuclear waste leaks (see examples below). The PRC further suggested that AST explore opportunities for recycling non‑biodegradable products within its material processing and manufacturing research.
Reduced fuel consumption and GHG emissions | Developed and promoted use of eco‑friendly products | Reduced other environmental risks and pollution |
---|---|---|
|
|
|
Conclusion and recommendations
Conclusion
Relevance
AST’s competitive advantage is its inclusion of both materials processing and manufacturing and vehicle and transportation systems engineering within one centre, and its focus on multiple modalities. However, it has yet to fully leverage this advantage to it and Canada’s mutual benefit as AST’s two research sections operate largely independent of one another.
As is, AST, with its recognized expertise and large scale facilities, stands out as unique in Canada, though not internationally. Each section’s research is aligned with the priorities of industry and of the federal government, but project selection has tended to be more reactive than strategic. AST has opportunities to respond to technological disruptions that intersect both the manufacturing and transportation sectors, to more directly contribute to Canada’s prosperity, and to actively consider end‑users.
Engagement
AST has room to grow its profile and engage more with Canadian industry and the research community. It has employed some successful engagement strategies, including its R&D group model for sectoral engagement, and participation in government and industry committees. Awareness and engagement have been relatively stronger for the materials/manufacturing section, compared to the transportation engineering section. Limited engagement by the latter section, particularly with key leaders in academia, will inhibit AST’s entry into new and emerging areas. Greater engagement may raise AST’s profile and lead to opportunities to leverage third‑party funding and capabilities.
Scientific excellence
AST has advanced scientific and technical knowledge in material processing and manufacturing, and in niche areas of transportation engineering (e.g., rail) through technology deployment and optimization. It has the potential to lead in advanced ground transportation systems if it addresses the relevance, engagement, and resource issues described above.
Resources
AST has had sufficient resources to meet client needs to date, and has been building and transitioning its capabilities for R&D in new and emerging areas. However, it has been constrained by budget limitations and competing priorities (e.g., revenue targets), and some HR strategies have not panned out (e.g., retire‑and‑replace). As a result, AST lacks the critical mass of expertise and facilities needed to move into new and emerging areas. It will need to strategically make investments that contribute to its competitive advantage, and find more opportunities to leverage the capacities of third‑parties.
Impact
AST’s is contributing to economic, environmental, and social impacts, as well as to federal policy solutions. It has supported innovative businesses for growth as its industry clients have adopted new technologies, launched new products, and increased their competitiveness. AST is also creating new opportunities for Canadian researchers and businesses, particularly in advanced manufacturing, through its industrial R&D groups and the NGen cluster (e.g., new partnerships, increased innovation capacity). With OGDs, transit agencies, and other stakeholders, AST has contributed to a more safe, secure, and reliable transportation systems, particularly to the benefit of vulnerable populations. Overall, half of AST’s clients are actively working towards a cleaner environment, including decreasing pollution and increasing the use of more eco‑friendly products. Finally, AST has delivered evidence‑based solutions to inform government decisions in priority areas such as transportation and defence.
Recommendations
Recommendation 1:
AST should articulate a plan that clarifies and promotes the integration of its capabilities in both material processing and manufacturing, and vehicle and transportation systems engineering, and pilot strategies to support this integration. This plan should identify where AST wants to be a leader (and commit resources accordingly), opportunities and risks for Canada posed by mutual disruptions to the advanced manufacturing and ground transportation sectors, and how it will contribute to Canada’s competitive advantage in these sectors.
Rationale: AST has yet to fully exploit its breadth of research, namely its inclusion of both manufacturing and transportation domain capabilities within one centre. Moreover, it has tended to be reactive to client needs rather than proactively identifying and targeting issues of strategic importance to industry. AST has an opportunity to strengthen its position—and potentially that of Canada—if it were to leverage its other unique characteristics (i.e., recognized expertise, large scale facilities) and further integrate its capabilities.
Recommendation 2:
AST should increase awareness among and engagement with key stakeholders in new and emerging fields. In particular, it should diversify and broaden its customer base in vehicle and transportation systems engineering, including greater collaboration with key players such as universities in order to access their competencies and facilities.
Rationale: AST’s visibility among and engagement with stakeholders could be greater, particularly in the research community. It will need to engage with a broader range of clients and partners to effectively implement its proposed ATSP and enter new fields where it wishes to be a leader.
Recommendation 3: AST should identify and implement strategies to address talent and facility gaps, particularly for its work related to advanced transportation systems. In addition to operationalizing existing HR and facility strategies, AST should explore new strategies to increase capacity to ensure it successful entry into new fields.
Rationale: While AST has had sufficient resources to meet client needs to date, it lacks the critical mass or expertise needed to move into new and emerging areas. Across sections, capital investments are needed to address emerging client needs and to position AST as a research leader. It is in a critical point as it is required to reduce its spending at the same time it intends to enter new areas. In order to enact their vision for the ATSP, it will need to set priorities and be strategic with its resources.
Recommendation 4:
AST should determine and communicate strategic indicators that measure its contribution to the Canadian economy, demonstrate scientific excellence (using more industry‑relevant metrics), and track influence on government operations, policy, and regulations.
Rationale: The PRC found that AST has opportunities to better track its achievements and, in doing so, demonstrate leadership, and attract new business, partners, and talent. For instance, it demonstrated scientific excellence through technology deployment and optimization rather than scientific publications. Moreover, its contributions to economic, environmental, and social impacts, as well government policy solutions, were observed by the evaluation but have been insufficiently documented.
Recommendation | Risk‑level Associated with not Addressing Recommendation | Management Response | Measure of Achievements | Proposed Person(s) Responsible | Expected Date of Completion |
---|---|---|---|---|---|
1. AST should articulate a plan that clarifies and promotes the integration of its capabilities in both material processing and manufacturing, and vehicle and transportation systems engineering, and pilot strategies to support this integration. This plan should identify where AST wants to be a leader (and commit resources accordingly), opportunities and risks for Canada posed by mutual disruptions to the advanced manufacturing and ground transportation sectors, and how it will contribute to Canada’s competitive advantage in these sectors. | High | Response: Accepted | Director General, AST | September 2020 | |
Action 1: Establish a new internal strategic advisory committee and its governance to increase collaboration across the Automotive and Surface Transportation (AST) sites. | Strategic advisory committee and governance established by the release of Terms of Reference. | ||||
Action 2: Define the value proposition of the Advanced Transportation Systems Program (ATSP). | ATSP plan approved by the Transportation and Manufacturing Division Vice‑President. | ||||
Action 3: Identify area of synergies in both material processing and manufacturing and vehicle and transportation systems engineering. | Project plans developed for two projects. | ||||
Action 4: Revise AST strategic plan to better align with all four recommendations of the evaluation. | Revised version of the AST Strategic Plan approved by the Transportation and Manufacturing Division Vice‑President. | June 2021 | |||
2. AST should increase awareness among and engagement with key stakeholders in new and emerging fields. In particular, it should diversify and broaden its customer base in vehicle and transportation systems engineering, including greater collaboration with key players such as universities in order to access their competencies and facilities. | Medium | Response: Accepted | Director General, AST | September 2021 | |
Action 1: Revise stakeholder’s engagement plan for the Advanced Manufacturing Program (AMP) and ATSP with an additional focus on vehicle and transportation systems engineering. | Stakeholder engagement plans for AMP and ATSP approved by AST Director General. One new multi‑partner project per program (including academia involvement) initiated. | ||||
A minimum of two major marketing initiatives completed as the rebranding of the Research Centre and/or sites, corresponding to 25% of the completion of the marketing plan. | |||||
Action 2: Identify area of synergies in both material processing and manufacturing and vehicle and transportation systems engineering. | Project plans developed for two projects. | ||||
3. AST should identify and implement strategies to address talent and facility gaps, particularly for its work related to advanced transportation systems. In addition to operationalizing existing HR and facility strategies, AST should explore new strategies to increase capacity to ensure it successful entry into new fields. | High | Response: Accepted | Director General, AST | June 2021 | |
Action 1: Reassess critical expertise and facilities gaps related to AST. | Lester Road strategic plan with a specific section on competency requirements, gaps, and strategies developed and approved by DG and presented to the Transportation and Manufacturing Vice‑President. | ||||
Action 2: Explore options to increase investment for talent and facility gaps in AST, internally and externally. | Identified and documented ways to acquire additional talent and capital investments. | ||||
4. AST should determine and communicate strategic indicators that measure its contribution to the Canadian economy, demonstrate scientific excellence (using more industry-relevant metrics), and track influence on government operations, policy, and regulations. | Low | Response: Accepted | Director General, AST | June 2021 | |
Action 1: Develop an AST Performance Framework in collaboration with Policy, Strategy and Performance (PSP) Branch including a Management Toolkit. | AST Performance framework completed and approved by AST management committee. | ||||
Strategic indicators monitored at established frequency. | |||||
Action 2: Develop a post‑project consultation to collect quantitative metrics and qualitative evidences on the impact of AST’s work on the on clients/collaborators, government policy solutions, and / or scientific excellence. | Post‑projects consultation developed and implemented. | ||||
Action 3: Develop a tracking tool for participation in committee and conferences to track influence. | Documented participation in committee and conferences. |
Appendices
Appendix A – Methodology
Bibliometric study
NRC’s Library and Information Management Services conducted a bibliometric analysis of peer‑reviewed publications affiliated with the research centre and indexed in Scopus for the period 2013‑2018. This analysis was used to assess the research centre’s impact, as well as to benchmark it against the NRC, Canada and the world. The list of publications and patents was developed in consultation with AST.
Key informant interviews
Interviews were conducted with 52 stakeholders (31 internal and 21 external) to collect information such as personal experiences, opinions and expert knowledge related to the relevance and performance of AST. This includes interviews with clients (n=15) to compliment the client survey as well as external stakeholders such as industry associates (n=6), split proportionally between AST’s materials/manufacturing and transportation engineering sections. This information was used to complement other lines of evidence and to contextualize quantitative information.
Client survey
A web‑based survey of AST’s clients was conducted to assess AST’s relevance, engagement, resources and impact. A total of 128 clients responded, including 80 that worked with the materials/manufacturing section, 39 for transportation engineering, and 9 that worked with both; rates proportionate to AST’s total client population. An external consultant conducted follow‑up calls to increase the response rate, which was 28%, a rate in line with industry standards.
Document and literature review
Internal and external documents were reviewed to provide context and to complement other lines of evidence in assessing relevance and performance. Internal documents included program business plans, RC operational and strategic plans, and presentations. External documents included industry studies, market assessments, and documents related to Government of Canada priorities.
Data review
Research centre and program administrative and performance data for 2013‑14 to 2018‑19 were reviewed to provide information on program inputs (i.e., resources), outputs, and client reach. This included financial data, human resource data, project data and intellectual property data.
Peer review
A peer review committee (PRC) was convened to assess AST along five dimensions (relevance, engagement, resources, scientific excellence, and impact), and identify ways AST can integrate two research sections. The PRC included eight members plus one chair, and balanced individuals with expertise related to materials processing and manufacturing, and vehicle and transportation systems engineering. They included national and international representatives from academia and industry. To ensure objectivity and avoid conflicts of interest, peer review committee members signed a confidentiality and conflict of interest agreements. The process included:
- Reviewing background material produced by AST and by the NRC evaluation team.
- Participating in a pre‑site visit teleconference to discuss their initial assessment of the RC, information gaps and questions.
- Participating in a two‑day site visit to the NRC.*
- Writing a peer review report.
*The chair and members with transportation engineering expertise also participated in a one‑day visit to the NRC‑Ottawa for the concurrent facility review (see next page).
Comparing AST by Research Area
AST’s research covers a variety of areas, from material processing and manufacturing (applied to the automotive sector), to vehicle and transportation systems engineering (including rail, heavy duty vehicles, and defence fleets). As a result, data were analyzed according to, and findings are reported by, these two broad areas. This was done to compare particularities or nuances in the work of AST relative to their objectives, rather than to rank each section of AST’s relative performance. A comparison of performance would be flawed as there are differences in the work AST conducts in materials/manufacturing and transportation engineering. To conduct these analyses, data was split when possible at the per project level (e.g., revenues and expenses, program outputs). In some cases however, data splits were not as precise and proxies were employed (e.g., HR analyses were based on locations, each of which mostly aligns with one research area over another).
Ongoing Facilities Review
In addition to this evaluation, AST has received, and will receive additional recommendations and advice regarding its facilities, namely through the NRC‑wide Facilities Review process. Transportation engineering facilities were reviewed concurrent to this evaluation, but materials/manufacturing facility reviews are still ongoing. Where possible, evidence from the facility review was incorporated into the evaluation’s lines of evidence. Self‑assessments completed by TEC management were included in the document review, and, as mentioned above, some peer reviewers participated in an additional day of site visits and discussions in Ottawa, giving them extra insights into AST’s operations.
Limitations and mitigation strategies
Challenges associated with PRC
To provide an objective, independent assessment of the RC, arms‑length experts were sought to participate in the peer review process. Given the need to be independent and objective, experts selected for PRC may not have in‑depth knowledge of the RC and its activities, or the NRC. Additionally, each member comes to the exercise with their own experience, expertise and associated biases.
In order to ensure the needed expertise on the committee, and to select the best possible committee composition, the project team consulted with the RC’s senior management and advisory board, and with the Knowledge, Information and Technology Services branch, to identify and invite the peer review chair and members. All PRC members were vetted and approved by the research centre DG and VP.
To mitigate any bias throughout the peer review process, the evaluation team tries to ensure the inclusion of experts from various areas of expertise, including different genders, geographic locations, and representatives from various types of organizations. In addition, at least one Canadian representative is sought in order to bring an understanding of the Canadian context.
Data and document availability
AST did not systematically track its outreach activities, such as conference or trade show attendance. To mitigate this gap in data, interviews with internal staff were used to gather examples. In addition, AST did not systematically track professional development of its staff. Instead, time spent on internal R&D projects was used as a proxy for competency development.
Use of publications to measure excellence
The challenge with bibliometric analysis is that there is a time lag of citation of published work. As a result, the actual use of more recent publications is likely underestimated in the current study. To mitigate this limitation, other lines of evidence were used to assess the excellence and scientific impact of AST’s research, including the use of an expert peer review.
Peer Review Committee Members
Expertise in material processing and manufacturing
Abhay (Abe) Vadhavkar, Chair
Former (recently retired) Director, Materials and Manufacturing Technology – Centre for Automotive Research
Thomas Kelly
Executive Director and CEO – Automation Valley
Tim Eden
Head of the Materials Processing Division, the Applied Research Laboratory and Professor – Pennsylvania State University
Maria Auad
Director, the Center for Polymers and Advanced Composites and Professor, Center for Bioenergy and Bioproducts, Engineering Group – Auburn University
Expertise in vehicle and transportation systems engineering
Yuping He
Professor – University of Ontario Institute of Technology
Donald Eadie
President – Don Eadie Consulting
Ahmed Shabana
UIC Distinguished Professor and Richard and Loan Hill Professor of Engineering – University of Illinois at Chicago
Séamus Parker
Principal Researcher, Transportation – FPInnovations
Marianne Hatzopoulou
Associate Professor, Canada Research Chair – University of Toronto Transportation Research Institute
Appendix B – AST Logic Model
Ultimate (10+ years) | Economic Impacts:
|
Environmental Impacts:
|
Social Impacts:
|
Intermediate (6‑10 years) | Business Innovation:
|
Advancement of Knowledge:
|
Government Policy Solutions:
|
Immediate (1‑5 years) | Business Innovation:
|
Advancement of Knowledge:
|
Government Policy Solutions:
|
Outputs | People Development, Infrastructure, Firms Supported, Scientific Achievements, and Collaborations. | ||
Activities | Facilitate Scientific, Technical & Organizational Capacity; Build Scientific & Technical Capacity; and, Deploy Scientific & Technical Capacity | ||
Inputs | NRC Investment, Earned Revenues, Contributions from other NRC Research Centres & Collaborators/Partners, NRC Common Services & SSC, HQP, and Facilities. |