Summary Report of Evaluation of NRC Ocean, Coastal and River Engineering (OCRE) Portfolio - August 2016

Relevance

Continued need for OCRE – The industry sectors targeted by OCRE are of strategic importance to Canada. These sectors include defence, marine energy and hydropower, shipbuilding and ship design, ports infrastructure and marine transportation, defence and offshore oil and gas.

Within these sectors, there are ongoing needs that OCRE is well positioned to meet. By providing expertise, facilities and an independent third party assessment, OCRE meets the needs of its stakeholders. Within Canada, there are instances of some overlap in competencies at OCRE and in other Canadian organizations (i.e., private sector and universities). There are potentially missed opportunities for OCRE to leverage the strengths of these organizations and present a common Canadian front to capture a greater market share.

While these other national organizations do not have the facilities that OCRE has, other international organizations do. In this regard, their needs could be met in the absence of OCRE; however, it would be at a competitive loss to Canada and Canadian strategies in the offshore, north and arctic.

Alignment with federal government and National Research Council (NRC) priorities - OCRE's strategic plan is aligned with NRCs strategic outcome of "Canadian businesses prosper from innovative technologies". Likewise, OCRE's strategic objectives and activities are aligned with federal government priorities related to Canada's North, shipbuilding, environmental responsibility and economic prosperity.

Alignment with other government departments - OCRE's facilities and competencies do not overlap with those of other government departments (OGDs), and rather are complimentary. OCRE has a track record of collaboration with all federal departments with mandates related to OCRE's areas of activity. However, opportunities for further collaboration were identified.

Appropriateness of the federal role - There is no federal marine/offshore strategy at this time – which makes an economic strategy for NRC more difficult. However, the federal government has a role to play in the marine industry. Various factors made the federal role appropriate, including the pre-commercial nature of some technologies as well as the importance of the industry to the federal government and Canadians. NRC also has a role of supporting federal regulators. Work done at OCRE includes e.g. supporting Transport Canada. In addition, the need for a neutral third party service provider, and the private sector's inability and unwillingness to take on the role due to the expensive, cyclical nature of the work was found to require federal intervention.

Performance

Client engagement strategy, awareness and reach - Client engagement is a high priority in OCRE's strategy and is consistent with its mandate and operational model. However, in the execution of OCRE's client engagement activities, internal staff noted that there was potential for confusion about the respective roles and responsibilities of the OCRE portfolio and Business Management Support (BMS) staff.

OCRE established its reputation as a testing facility. However according to a few external interviewees OCRE was moderately successful at increasing awareness of its research expertise and technological development capabilities in a number of areas (i.e. arctic research). Still external interviewees also pointed to some improvements in outreach.

Over the evaluation time period, there was a decrease in revenues, number of projects and clients. While the decline in revenues was largely due to a decreased number of projects with OGDs, also influencing OCRE's revenues were market conditions (e.g., decline of the oil and gas sector). Despite efforts to increase OCRE's national presence beyond Ontario, and Newfoundland and Labrador, it remains an area for improvement. Regionally, since the NRC transformation, OCRE has had to overcome the expectations of the local sector for free services. Raising the cost for services has led to fewer local initiatives.

Client satisfaction and outcomes - The large majority of OCRE clients were very satisfied with the services they received, indicating that they would work with OCRE in the future as well as recommend OCRE to others. Clients also reported many positive outcomes as a result of their work with OCRE, in which they said OCRE played a vital or significant role.

Resource Utilization

Adequacy of human resource critical mass and competencies – OCRE had various processes in place to ensure adequate human resource critical mass and competencies. While OCRE generally had the competencies needed to meet the needs of its hosted programs, it did not have sufficient human resource critical mass to deliver on both internal and external projects. The OCRE Strategic Plan update (2016-2021) clearly states OCRE's intent to address this situation and management is taking a more strategic approach in 2015‑16.

Adequacy of scientific and engineering infrastructure - OCRE has the necessary infrastructure to deliver on its projects. In order to ensure that the portfolio had appropriate infrastructure to meet client needs, OCRE took a holistic, long-term view of its recapitalization efforts. This included using a coordinated approach to identify the infrastructure needs of the portfolio-hosted programs. Investment review/planning into new major research facilities is underway to meet the future needs of Canada's offshore sector.

Project management effectiveness - The evaluation identified opportunities to improve the effectiveness of project management at OCRE. OCRE's approach of using researchers as project managers was consistent with common practises in other similar international organizations and NRC portfolios. Project managers can also be the researcher and it does not have to be two separate people. However, OCRE did not have a defined approach to assign researchers to project management roles. In the majority of cases, researchers were not assigned to manage projects based on their project management competencies relative to project materiality and/or risk.

Process and tools to support project management - In order to facilitate effective project management practises within the portfolio, OCRE provided its staff with project management training and support (through a project management support office), and implemented project management processes and tools. Despite this, the evaluation identified opportunities for OCRE to improve its project management processes and tools related to change management, risk management, project monitoring and project close-out. OCRE had also established an approach to facilitate ongoing improvements to its project management processes and tools (i.e., via the portfolio Project Management Community of Practice). However, the evaluation found that improvements to OCRE's project management processes and tools had not yet materialized during the evaluation period. However, in 2015‑16, OCRE management started to address these issues to some degree by looking at projects needs and adopting separate technical and project management roles when appropriate. This project management approach is highlighted in the OCRE strategic plan.

Efficiency and effectiveness of operations – Portfolio operations were not as efficient as originally targeted by the portfolio; OCRE did not achieve its targets for overhead efficiency, staff utilization or facility utilization over the evaluation time period (noting that the NRC cyber intrusion caused significant restrictions to OCRE's operations). Operational efficiency also decreased over the evaluation time period. While an extensive review of the NRC Common services was not conducted as part of this evaluation, the evaluation did reveal that there were opportunities for KM and DFS, in particular, to ensure more efficient and cost-effective operations. Despite these operational challenges, OCRE appears to be making progress toward achieving its objectives considering the portfolio's modest financial resources.