The NRC Construction Portfolio, formerly known as the NRC Institute for Research in Construction (NRC-IRC) up until April 2012, is Canada's national construction research and technology development agency. It aims to bring quality to the built environment by performing research and development, conducting product evaluation, and developing test methods, standards and decision-support tools. Through ongoing research, it helps support the marketplace by offering services and technology to public and private sector clients that improve the construction of high-performance buildings and infrastructure. Additionally, it leads the development of the National Building Code of Canada.
This report presents the results of the evaluation of the Construction Portfolio, when it operated as an Institute within the National Research Council of Canada. The evaluation was undertaken to assess the relevance and performance of NRC-IRC in order to provide NRC senior executives and managers with information that will contribute to improved program planning and results achievement in the future as well as to address the requirements of the TBS Policy on Evaluation (2009). The scope of the evaluation included the majority of NRC-IRC's construction-related activities from 2005-06 to 2011-12. While the evaluation did not directly cover the activities of the new Construction Portfolio, the findings presented in the report are expected to support decision-making in a period of transition by providing insights about strategic and operational challenges in areas that remain relevant to the new entity (i.e., the Portfolio).
The evaluation was led by an independent evaluation team within NRC's Office of Audit and Evaluation. A mixed methods approach was used for this evaluation. The approach allowed for triangulation and complementarity through the use of both qualitative and quantitative methods. The specific methods used in the study included: an internal and external document review; a literature review on performance-based and objective-based approaches to building regulations; a review of administrative and performance data; key informant interviews; and case studies.
Findings – Relevance
NRC-IRC has in the past fulfilled a large diversity of client needs in various areas. While the Institute has established the strategies and mechanisms to identify, prioritize and address the needs of its stakeholders, there is some evidence that additional opportunities exist within the existing areas of focus and other construction related sectors in Canada where NRC-IRC could offer services and support to meet the needs of industry and Government. In terms of the National Model Code, NRC-IRC, in collaboration with the Canadian Commission on Building and Fire Codes (CCBFC), has successfully developed a version of the National Model Code (NMC) that addresses the needs of the Provinces and Territories (P/T's). Opportunities for improvement were however identified with regards to the communication of NRC's new strategic orientation and programs to key collaborators from the federal government. In fact, these organizations expressed uncertainties with regards to NRC's ability to meet their current and future needs, primarily because of limited communication between NRC and their organization during NRC's strategic repositioning.
Recommendation 1: NRC should take the appropriate means to ensure that other government departments are aware and understand the new strategic direction of the organization and how the new NRC Programs and services can support their mandate.
Accepted; Since April 1, 2013, NRC Construction management has been implementing plans to communicate the new strategic directions to stakeholders, including other government departments, and how NRC programs, competencies, and facilities can support them in their mandate.
Appropriateness of Federal Role
There is a role for federal government involvement in support of construction-related research given the need of various stakeholders to access strong multidisciplinary R&D competencies to address complex problems faced by the industry. For historic reasons, these competencies have been located within NRC. This is a role that NRC-IRC is best positioned to play due to its unique expertise and facilities, its multidisciplinary skills, its impartiality and credibility, its ability to bring together various players for multi-party and multi-disciplinary initiatives, and its reputation for providing high quality and reliable data. NRC-IRC R&D activities were not duplicative to that of other federal or provincial government departments and were rather complimentary.
In terms of building codes, NRC-IRC is well positioned to support the code development process because, as a federal agency located within the central government, it can ensure the harmonization of standards and regulations across jurisdictions. It can also generate economies of scale by avoiding duplication of the code development process in each P/T. Moreover, the fact that NRC-IRC is a federal government associated Research and Technology Organisation (RTO) provides credibility to the National Model Code (NMC) and facilitates the use of scientific evidence to support code change.
Alignment with Federal Priorities
Moving forward, the objectives and future activities of the NRC-IRC, as NRC Construction, align well with NRC's new strategy and also with Government of Canada priorities. NRC Construction activities address five of the six critical issues for Canada that NRC intends to support under its new strategic direction, officially announced in 2012-13, including: economic development; pressures on natural resources; climate change; and a changing sense of communities. In terms of alignment with government priorities, the activities of NRC Construction are consistent with those that relate to: energy efficiency and climate change; major infrastructure; research and innovation for economic growth; security; and the health and safety of Canadians.
Findings – Performance
NRC-IRC has been successful at reaching clients from both the private and public sector through both collaborative R&D and fee for service work. Between 2005-06 and 2011-12, NRC-IRC undertook nearly $45 million of research related work for 283 unique clients as part of 453 projects. In addition to this, NRC-IRC provided product evaluation services to 795 unique clients. The Institute's reach to new clients decreased in the beginning of the period covered by the evaluation, however, generally stabilized in the later years. Despite this, NRC-IRC's reach to innovative private sector firms has not been fully exploited, due in part to the Institute's limited understanding of who the innovative firms in Canada were, where they were located and what their R&D needs were.
Recommendation 2: The new Construction Portfolio, should seek ways to increase its understanding of its potential targeted clients (i.e., innovative firms in the construction sector), including those outside of Ontario and Quebec.
Accepted; the management recognizes the broad benefits of increasing the knowledge and understanding of NRC's stakeholders in the construction sector. The new NRC Programs related to Construction are designed in a way that is conducive to support this recommendation. The management will ensure that it will be implemented appropriately and across Canada.
In addition to the collaborative work the Institute engaged in and the services it provided to clients, it also reached external clients through the sale of the national model building code. NRC-IRC was found to have mechanisms in place that facilitate increased reach, such as the use of consortia to conduct research, working with industry associations, and the Canadian Centre for Housing Technology (CCHT). However, the case study on CCHT showed that this facility will require additional investments in the near future in order to remain relevant to the industry.
Recommendation 3: Given that CCHT supports the needs of companies to assess the performance of their products and technologies, that it has contributed to NRC-IRC's ability to increase its outreach to the private sector and that further investment will be required to update the facility, it is recommended that the Construction Portfolio assesses the extent to which the facility is aligned with the strategic objectives of NRC , and if it is deemed a priority, work with CCHT stakeholders to invest the appropriate resources to ensure CCHT's continued relevance to industry.
Accepted; the management agrees with the need for a CCHT assessment respective to its alignment with the NRC programs and strategic direction. The outcome of this assessment and the anticipated program demands on CCHT will determine the way forward toward its future mode of operations as well as mid to long-term investments by NRC and the other CCHT stakeholder organisations.
National Model Codes
The presence of a NMC was found to be an efficient mechanism to ensure the harmonization of codes and standards in Canada. While the 2010 version of the NMC is expected to be adopted by almost all P/Ts, Ontario and Newfoundland are not planning to adopt this version of the code. The financial sustainability of the code development system depends on not only the adoption of the NMC, but also on the timely adoption.
Recommendation 4: In order to accelerate the adoption of the National Model Codes by all Provinces and Territories, it is recommended that the new Construction Portfolio:
- seek ways to increase the engagement of Provinces and Territories; and,
- take a leadership role in consulting with the CCBFC and PTPACC to coordinate/streamline the public consultation process.
Accepted in principal; the management agrees with the finding that a faster and comprehensive adoption of building codes would be a desirable outcome. It will therefore undertake efforts to find ways and take leadership as recommended. However, it must be noted that NRC has to fully respect all constitutional aspects of federal and provincial responsibilities.
In 2005, the CCBFC published the first objective-based version of the national model codes. This new approach to building regulations marked a major change to the regulatory construction system in Canada as before 2005, Canada's regulatory approach to building codes was prescriptive in nature. While instances of increased innovation resulting from object-based codes were identified, an intended outcome of the new regulatory framework, widespread adoption of innovative products and design in the built environment has not yet occurred. Two factors were largely found to explain why objective-based building codes have not yet resulted in increased innovation in the construction sector:
- training of building officials; and
- efficient product evaluation services.
NRC-IRC's R&D Activities
NRC-IRC's R&D activities have produced outputs that are highly diversified, reflecting the range of activities undertaken by the Institute in various research areas. Outputs have included knowledge creation, test results from compliance / performance assessments of products and materials, new test methods and protocols, industry guidelines and practical tools for industry, development of new/improved products, and technologies and tools in support of decision-making. Specific conditions and factors were identified as key determinants in the achievement of longer-term impacts for large scale R&D projects. These included the existence of appropriate knowledge and technology transfer mechanisms, the inclusion of key stakeholders in research consortia, code change and technology demonstration projects. Anecdotal evidence was found to suggest that NRC-IRC R&D activities have, in some cases, increased market access and growth of the industry, resulted in cost-savings for public sector organizations and homeowners, improved fire safety, and contributed to the health and quality of life of Canadians. Environmental benefits resulting from NRC-IRC's R&D activities are also expected but have not yet been realized.
Findings – Economy and Efficiency
NRC-IRC has used its resources in an economical manner between 2005-06 and 2011-12. The economical use of NRC-IRC resources was apparent from the stability of financial resources used by the Institute, the funds leveraged for research projects, the use of joint facilities (e.g., CCHT), and in the operating costs of its laboratories, which were lower than those of comparator laboratories.
Performance information on productivity as well as improvements made to the program delivery have shown an efficient use of its resources by the NRC-IRC during the period covered by the evaluation. Client satisfaction with historical program activities and outputs (i.e., services received, deliverables produced) was also used as an indicator of NRC-IRC's operational efficiency. Overall NRC-IRC clients were very satisfied with the services they received. However, there were some concerns raised with the timeliness of services and deliverables.
Recommendation 5: Given the importance of the client experience as part of the new NRC strategy and based on the finding that opportunities exist for NRC-IRC to improve the timeliness of its services and deliverables, it is recommended that the Construction Portfolio reviews ways to improve its service delivery.
Accepted; the management is confident that NRC's new business model based on diligent program and project management will lead to operational efficiencies and a timely provision of services and deliverables to clients. Program and project reviews will be undertaken to monitor this issue; as well as customer satisfaction surveys and follow up action.
The presence of sound management systems, processes and practises are factors that have positively affected the efficiency of NRC-IRC's operations. NRC-IRC also has made changes to the delivery of its activities to enhance efficiency over the evaluation time period and has steadily increased its productivity in more recent years (i.e., 2010-11 and 2011-12). While the Institute has reduced and eliminated the backlog at the Canadian Construction Material Centre (IRC-CCMC), opportunities still exist to improve the IRC-CCMC product evaluation services and product evaluation and decision-making process.
Recommendation 6: In order to optimize the efficiency of IRC-CCMC delivery, it is recommended that the Construction Portfolio implement the recommendations put forward in the 2010 IRC-CCMC Business and Governance Review.
Accepted; the management will undertake efforts to accelerate the implementation of the recommendations made in the 2010 review. Changes to NRC's new business model will be taken into account accordingly.
In terms of the NMC, the code funding model is an efficient system to ensure the development of the NMC on a five year cycle. It, however, only provides limited flexibility in terms of funding other types of activities that are not directly linked to the development of the NMC. Some of these activities have the potential to alleviate barriers to innovation in the construction sector.
Overall, the Institute has demonstrated a strong value for money for Canada as shown by its clear relevance and its ability to generate impacts in an efficient and economical way. Going forward, in light of the renewed NRC mandate to focus on the growth of the Canadian industry and support government department, NRC Construction's ability to engage key private and public sector partners on an on-going basis will be critical to the future success of the Portfolio.
Acronyms and Abbreviations
|BLEVE||Boiling Liquid expanding vapor explosion|
|CAA||Clean Air Agenda|
|CCA||Canadian Construction Association|
|CCBFC||Canadian Commission on Building and Fire Codes|
|CCHT||Canadian Centre for Housing Technology|
|CMHC||Canada Mortgage and Housing Corporation|
|CIS||Client Information System|
|CSIR||Centre for Sustainable Infrastructure Research|
|DND||Department of National Defense|
|EAC||Evaluation Advisory Committee|
|GDP||Gross Domestic Product|
|ICC||International Code Council|
|IRC-IRC-CCC||Institute for Research in Construction Canadian Code Centre|
|IRC-IRC-CCMC||Institute for Research in Construction Canadian Construction Material Centre|
|IRC-CSIR||Institute for Research in Construction Centre for Sustainable Infrastructure Research|
|IRCC||Inter-jurisdictional Regulatory Collaboration Committee|
|NAICS||North American Industry Classification System|
|NCM||National Model Codes|
|NRCan||Natural Resources Canada|
|NRC-ASPM||National Research Council Administrative Services and Property Management|
|NRC-IRC||National Research Council Institute for Research in Construction (referred to as the NRC Construction as of April 1st, 2012)|
|NRC Construction||National Research Council Construction|
|NRC-OAE||National Research Council Office of Audit and Evaluation|
|SEC||Senior Executive Committee|
|OGD||Other Government Departments|
|PTPACC||Provincial and Territorial Policy Advisory Committee on Codes|
|P/T||Provinces and Territories|
|R&D||Research and Development|
|SIGDERS||Special Interest Group for Dynamic Evaluation of Roofing Systems|
|TBS||Treasury Board Secretariat of Canada|
|UFI||Urea Formaldehyde Foam Insulation|
|VIP||Vacuum Insulated Panels|
|VOC||Volatile Organic Compounds|
The NRC Construction Portfolio, formerly known as the NRC Institute for Research in Construction (IRC) up until April 2012, is Canada's national construction research and technology development agency. It aims to bring quality to the built environment by performing research and development, conducting product evaluation, and developing test methods, standards and decision-support tools. Through ongoing research, it helps support the marketplace by offering services and technology to public and private sector clients that improve the construction of high-performance buildings and infrastructure. Additionally, it leads the development of the National Building Code of Canada.
The evaluation was led by an independent evaluation team from the NRC Office of Audit and Evaluation (OAE), with support from an external consultant for specific evaluation methods. The work of the evaluation team was also supported by an Evaluation Advisory Committee (EAC), which provided advice related to the evaluation framework, approach, interpretation of findings and recommendations. These experts did not act in a decision-making fashion, but rather played an advisory role in ensuring a high-quality and useful evaluation product.
Following the evaluation overview presented below, Section 2 of this report provides a profile of the institute, including the context within which it operates. The study's findings, which are organized by broad evaluation issues (relevance, performance, efficiency and economy), are discussed in Section 3 through Section 5. Section 6 of the report presents the general conclusions from the study while Section 7 of the report lays out management's response to the recommendations that were made and the actions that will be taken as a result of the evaluation.
1.1 Evaluation Overview
An evaluation of NRC-IRC was conducted in fiscal year 2012-13. The scope of the evaluation included most of NRC-IRC's construction-related activities from 2005-06 to 2011-12 with some exceptions.Footnote 1 While the evaluation did not directly cover the activities of the new Construction Portfolio, the findings presented in the report are expected to support decision-making in a period of transition by providing insights about strategic and operational challenges in areas that remain relevant to the new entity (i.e., the Portfolio).
The Centre for Sustainable Infrastructure Research (IRC-CSIR) in Regina was not included, since it had been evaluated twice over the past five years.Footnote 2 Likewise, NRC-IRC's Computer-Assisted Construction Technologies Program, located in London, was also excluded from the evaluation because NRC decided to consolidate the competencies in Ottawa as of March 31, 2012. Finally, the current evaluation examined the impacts of investments in the Institute's Indoor Environment Program, but did not address the relevance, efficiency and economy of this Program, given that these elements were largely addressed under the 2010 Evaluation of the Clean Air Regulatory Agenda led by Health Canada. Footnote 3
1.1.1 Evaluation Rationale
The evaluation was undertaken to assess the relevance and performance of NRC-IRC in order to provide NRC senior management with information that will contribute to improved program planning and results achievement in the future as well as to address accountability requirements of the TBS Policy on Evaluation (2009).
The specific evaluation questions, presented in Table 1, were selected based on consultations with program management and a review of key documents during the planning stage. The evaluation issues identified are also aligned to the requirements of the 2009 Treasury Board Policy on Evaluation. The complete evaluation matrix, which identifies the lines of evidence employed to respond to each question, is presented in Appendix A.
Table 1. Evaluation Issues and QuestionsFootnote 4
|Core Evaluation Issues||Evaluation Questions|
|Continued Need for Program||R1. What are the needs of NRC-IRC stakeholders?|
|R2. To what extent are the activities of NRC-IRC aligned to current and future needs of the Construction Portfolio's stakeholders?|
|Alignment with Federal Roles and Responsibilities||R3. To what extent are the activities of NRC-IRC consistent with federal roles and responsibilities?|
|R4. Is there a justifiable role for the federal government and, specifically, NRC in supporting the development of construction code?|
|Alignment with Government and NRC Priorities||R5. To what extent are NRC Construction's activities aligned with federal government priorities and NRC's strategic outcomes?|
|Achievement of expected outcomes||P1. To what extent and in what conditions has NRC-IRC been successful at reaching its targeted clients and collaborators?|
|P2. To what extent and in what conditions has NRC-IRC contributed to the harmonization of Construction Codes and accepted practices across Canada?|
|P3. To what extent and in what conditions has NRC-IRC contributed to the commercialization and/or increased market access for Canadian products and technologies?|
|P4. To what extent and under what conditions has the use/adoption of objective-based construction codes increased the use of innovative design or the adoption of innovative material and technology in the Canadian built environment?|
|P5. To what extent and in what conditions have NRC-IRC's activities contributed to the health, safety and accessibility to building for Canadians?|
|Demonstration of efficiency and economy||P6. To what extent has NRC-IRC used its resources in an efficient and economical manner in producing outputs and progressing towards expected outcomes?|
|P7. What factors have affected the extent to which NRC-IRC operates in an efficient manner?|
|P8. To what extent do code-related complaints affect NRC-IRC's operational efficiency? Does this vary by type of complaint, group or other key conditions?|
1.1.2 Evaluation Design and Methodology
The overall evaluation approach and level of effort were determined based on the resources available to support the project and a risk assessment conducted as part of the planning phase of the evaluation. The evaluation methodology integrated the use of multiple lines of evidence and complementary research methods as a means to enhance the reliability and validity of the information and data to be collected. The specific methods used in the study included:
- Internal and external document review;
- Literature review on performance-based and objective-based approaches to building regulations;
- Administrative and performance data review;
- Key informant interviews (Internal interviewees: n = 8; external interviewees: n =33); and
- Case studies (n = 7). Footnote 5
For key informant interviews, the following scale is used in this report to indicate the relative weight of the responses for each of the respondent groups:
- "All/almost all" – findings reflect the views and opinions of 90% or more of the respondents commenting on that particular issue;
- "Many/most" – findings reflect the views and opinions of at least 50% but less than 90% of the respondents commenting on that particular issue;
- "Some/several" – findings reflect the views and opinions of at least 25% but less than 50% of the respondents commenting on that particular issue; and,
- "A few" – findings reflect the views and opinions of at least two respondents but less than 25% of the respondents commenting on that particular issue.
A more detailed description of the study methodology, limitations and challenges is provided in Appendix B.
2. NRC-IRC Profile
NRC-IRC was originally established on August 1, 1947 as the Division of Building Research. As it evolved its focus in response to changing industry expectations, it later became NRC-IRC in 1986. It remained under this name until April 2012, when it was re-organized and re-branded as the NRC Construction Portfolio. The transition from an Institute to a Portfolio in 2012 corresponded to an NRC-wide corporate transformation with the goal of increasing the impact of the organization on the growth of the Canadian industry.Footnote 6
2.1 Objectives and Expected Outcomes
According to NRC-IRC's business plans for the period covered by the evaluation, NRC-IRC's vision was to be a recognized leader in the development of a quality-built environment through research, innovation and the creation of integrated solutions. NRC-IRC strived to increase the competitiveness and productivity of the construction and manufacturing industries and enhance living standards for Canadians by ensuring healthy, safe and accessible buildings. In order to do so, NRC-IRC: performed research and development; conducted product evaluation; developed test methods, standards and decision-support tools; and led the development of the National Building Code of Canada.Footnote 7
Between 2005-06 and 2011-12, NRC-IRC managed several facilities in four geographical locations: London, Ottawa and Almonte (all in Ontario), as well as Regina, Saskatchewan. The Institute was organized according to three platforms, which included:
- Platform 1: Research and Technology Development: As a multi-party collaborative research organization, NRC-IRC assumed a leadership role on issues such as infrastructure rehabilitation, service life-cycle prediction, product evaluation, fire safety, and health and quality of the indoor environment.
- Platform 2: Codes and Standards: NRC-IRC played a national leadership role in developing construction regulation by producing model building and fire codes and guides of practice. It supported technical standards and it facilitated a uniform and nationally integrated code development process.
- Platform 3: Information Dissemination: Using knowledge gained through the R&D conducted under platform one, NRC-IRC synthesized and disseminated knowledge and technologies to industry to meet the mid-and-long-term needs of Canada's construction industry.Footnote 8
NRC-IRC responded to evolving issues affecting the construction sector through six main programs. The historical suite of NRC-IRC's programs included:
- Building Envelope and Structure: This program developed technologies for design, construction and operation of all types of building systems, including both new construction and repair or renovation.
- Fire Research: Through research, this program developed technologies that would reduce the total cost of fire in Canada. While primarily directed towards buildings, this program also focused on the safety of Canadians in other environments.
- Indoor Environment: This program integrated existing NRC competencies to provide technologies to design and operate indoor environments that maximized the comfort, productivity, health and safety of building occupants. This program included the activities funded under the Clean Air Agenda, a cross government initiative designed to make improvements in Canada's environment by addressing the challenges of climate change and air pollution. The Clean Air Agenda was recently renewed in 2011-12 until 2015-16.
- Urban Infrastructure: This program focused on its research and technology transfer efforts on increasing the durability of urban lifelines and improving municipal asset and maintenance management practices. The Centre for Sustainable Infrastructure Research (CSIR) in ReginaFootnote 9 operated under this program.
- Codes and Evaluation: Under this program were the Canadian Codes Centre (IRC-CCC) and the Canadian Construction Materials Centre (IRC-CCMC). The IRC-CCC provided technical and administrative support to the Canadian Commission on Building and Fire Codes (CCBFC) and its related committees, which are responsible for the development of the National Model Construction Codes of Canada. IRC-CCMC offered an evaluation service for all types of building construction materials, products, systems and services. IRC-CCMC technical evaluations were based on the requirements of the National Building Code of Canada or Provincial/Territorial Building Codes.
- Computer-Assisted Construction Technologies: This program focused on developing and applying information and communication technologies to support the Canadian construction industry (e.g., integration technologies for critical facilities management and maintenance).
In addition to these six programs, NRC-IRC operated the Canadian Centre for Housing Technology (CCHT), a partnership of NRC-IRC, Canada Mortgage and Housing Corporation (CMHC) and Natural Resources Canada (NRCan). This is a role continued by the new Construction Portfolio (NRC Construction). CCHT supports the housing sector through research and evaluation of innovative housing technologies through side-by-side comparisons in two research houses. While NRC-IRC was the operating agent of CCHT, all three stakeholders managed the facility.
While outside of the focus of the current evaluation, it is worthwhile to note that as of April 2012, when the Institute became a Portfolio (NRC Construction), the NRC Programs were repositioned to address specific industrial sector needs. Similar to NRC-IRC's mandate, NRC Construction aims to develop and validate cost-effective technologies and solutions; develop and deploy national model building codes and regulations; and provide evaluations for new products and market access services. The Portfolio's leads the following NRC Programs ,but staff from the Portfolio support several others across NRC such as Working and Travelling on Aircraft, and NRC Arctic:
- High performance buildings: This program will develop and support the commercialization of retrofitted energy-efficiency technologies for commercial and institutional buildings.
- Building regulations for market access: This program will develop and diffuse uniform building regulations and solutions to reduce compliance costs and address societal requirements.
- Canadian wood in mid-rise buildings: This program will develop, improve and support the adoption of wood-based building products and assemblies for the 5-12 story building market.
- Critical concrete infrastructure: This program will develop and deploy through companies, high performance materials and technologies to reduce rehabilitation costs of concrete highway bridges, and extend their service life and shock resistance.Footnote 10
NRC-IRC was part of NRC's Engineering Portfolio and was managed by a Director General (DG). Within the Institute, each program was managed by a research Director who reported to the DG of NRC-IRC. Although NRC-IRC was supported by NRC Corporate for specific operational functions such as Human Resources, Communication and Finance, most of these operational activities were decentralized. The activities of NRC-IRC were also overseen by an external advisory committee.
This section presents the financial and human resources of NRC-IRC between 2005-06 and 2011-12, the period covered by the evaluation. It includes resources provided to NRC as part of the NRC Technology Cluster Initiative, the London Computer-Assisted Construction Technologies Program and the Clean Air Agenda. Although the evaluation did not examine the relevance and performance of this funding in detail, the inclusion of these resources provides the reader with a complete profile of the Institute.
2.4.1 Financial Profile
Between 2005-06 and 2011-12, NRC-IRC expended an average of $30.985M per year, which included a combination of A-base and B-base appropriations as well as revenue.Footnote 11 There was very little fluctuation in NRC-IRC expenditures over the seven year period (see Figure 1).
On average between 2005-06 and 2011-12, 67% of the Institute's expenses were spent on salaries and wages, while 25% were spent on operating and 8% on capital expenses. The operating expenditures of the Institute were largely influenced by the Code and Evaluation Program. The Code and Evaluation Program has operating expenses double to that of other programs because it pays for the travel expenditures (in accordance with Treasury Board Guidelines) of the CCBFC committee members, who normally meet several times a year in support of the development of the National Model Construction Code.
Figure 1. NRC-IRC Proportional Expenditures by Source and Fiscal Year
Long description of NRC-IRC Proportional Expenditures by Source and Fiscal Year
|Type of expenditure||Source of funding||2005-2006||2006-2007||2007-2008||2008-2009||2009-2010||2010-2011||2011-2012|
|Salary and wages expenditures||A-Base||54%||51%||52%||49%||56%||53%||48%|
Source: NRC-IRC Administrative Data
Figure 2 provides an overview of the Institute's expenditures by program between 2005-06 and 2011-12. The Code and Evaluation Program accounted for the largest proportion of NRC-IRC's average total expenditures at 24%, while the Computer Assisted Construction Technologies in London, Ontario, accounted for the lowest at 8%.
Figure 2. NRC-IRC Total Expenditures by Program and Fiscal Year (in Thousands of Dollars)
Long description of NRC-IRC Total Expenditures by Program and Fiscal Year (in Thousands of Dollars)
|Code and Evaluation||5,734.60||5,851.20||5,838.40||6,385.10||7,034.40||6,300.90||5,979.90|
|Building Envelope and Structure||4,538.70||4,036.30||4,610.80||4,726.90||4,775.20||4,278.10||3,950.70|
|Computer Assisted Construction Technologies||2,423.10||2,197.90||2,044.00||2,013.20|
Source: NRC-IRC Administrative Data
Note: The Computer Assisted Construction Technologies program only began in 2008-09.
Between 2005-06 and 2011-12, for each dollar of A-base expended, the Institute generated an average of $0.53 in revenue from the public and private sector. The greatest proportion of NRC-IRC revenue was generated from fee-for–service work and the sale of goods and information products to external clients (see Figure 3). This is largely attributable to the Code and Evaluation Program, which is responsible for the sale of the National Model Building codes and the product evaluation services provided by the Canadian Construction Materials Center (IRC-CCMC), which is housed in the Code and Evaluation Program. According to data from the IRC-CCMC client database, IRC-CCMC generated approximately $7.4M total revenues between 2005-06 and 2011-12.
Figure 3. NRC-IRC Revenues Generated by Source and Fiscal Year (in Thousands of Dollars)
Long description of NRC-IRC Revenues Generated by Source and Fiscal Year (in Thousands of Dollars)
|OGD||Royalties and license revenue||0.00||17.40||0.00||0.00||0.00||0.00||0.00|
|Fee for service||101.00||13.30||141.30||29.50||3.60||33.80||34.10|
|Sale of goods and information products||87.40||145.10||66.30||28.90||36.00||132.20||121.70|
|External clients||Royalties and license revenue||191.80||705.20||1,796.00||797.00||1,131.60||463.10||276.20|
|Rental of machinery and equipment||0.00||0.00||7.50||20.80||19.70||28.80||18.70|
|Fee for service||4,070.50||2,903.90||2,350.20||1,988.60||2,012.70||2,757.10||4,131.60|
|Sale of goods and information products||2,858.20||3,004.80||2,123.00||2,687.40||2,470.00||3,305.10||3,693.40|
Source: NRC-IRC Administrative Data
Of the revenue generated by the Institute between 2005-06 and 2011-12, on average, the majority (78%) was from external clients, with the remaining from OGDs (Figure 3). When the Code and Evaluation Program is excluded from the calculation of revenue generated by external clients, a different trend emerged. Specifically, the average revenue from external clients drops to 58 percent.
When revenues are examined at the program level, the Code and Evaluation Program generated the largest proportion of revenue from external client per year (average = 92% of revenue was from external clients) while the Indoor Environment Program generated the least per year (average = 41% of revenue was from external clients).
2.4.2 Human Resource Profile
The activities of NRC-IRC were delivered by an average of 250 staff each year. As shown in Figure 4, the number of human resources increased from 2005-06 to 2010-11 and was at its highest in 2008-09 and 2009-10 during the evaluation period. On average, the majority of the human resources were research staff (43%), followed by technical staff (27%), administrative support (23%), students (4%) and management (3%). The average ratio of technical staff to research staff was 1:0.62.
Figure 4. NRC-IRC Human Resource Profile from 2005-06 to 2011-12 (Number of Full Time Equivalents)
Long description of NRC-IRC Human Resource Profile from 2005-06 to 2011-12 (Number of Full Time Equivalents)
|Students (incl. co-oop students)||8||8||10||15||18||10||2|
Source: NRC-IRC Administrative Data
On average, the Urban Infrastructure Program and the Code and Evaluation Program employed the largest number of staff per year. The Code and Evaluation Program had the largest proportion of administrative staff compared to other programs on average per year between 2005-06 and 2011-12 (i.e., 43% versus 6% to 18% for other programs). This is largely attributable to the coordination required for the administration of CCBFC (e.g., administrative functions related to code publication and selling, maintenance of membership data and a nomination database for future members of the CCBFC).
2.5 NRC-IRC Context - The Canadian Code Development System
The idea of publishing a national building code can be traced to 1937, when the federal Department of Finance asked NRC to develop a model building code that could be adopted by Canadian municipalities. The intent was to harmonize the building practices across the country and to avoid major catastrophes related to fire or building failure. It was not until 1941 that the first version of the National building code was published by NRC. Although the code development system has experienced many changes over the years (e.g., creation of various consultation bodies, the development of the fire, plumbing and energy codes), the federal government, and more specifically NRC, has continued to play a key role in the system.
In Canada, the safety of buildings and the health of occupants rest under the authority of the provinces and territories (P/Ts). However, for historical reasons, the P/Ts have, for the most part, relied on the National Model Codes to provide the basis from which to regulate issues of safety and health for the buildings located in their jurisdictions. This section presents an overview of the Canadian code development system by outlining the roles and responsibilities of the main actors involved in the system.
2.5.1 Key Actors Involved in the Development of the National Model Codes
The Canadian Commission on Building and Fire Codes (CCBFC) as well as NRC-IRC are at the heart of the Canadian code development system (see Figure 5). The Commission, a neutral not-for-profit organization, develops the National Model Codes (NMC) through a consensus-based process that relies on the voluntary contributions of public and private sector experts from across Canada.Footnote 12 The system is structured such that it is the members of the Commission who establish the content of the model codes. Under the auspices of the Commission, 11 standing committees review and develop proposed technical changes to the codes before they are submitted for public review. The changes must then be approved by the CCBFC before NRC publishes the six model codes in English and in French.Footnote 13
Figure 5. Key Actors Involved in the Development of the National Model Code
Long description of Key Actors Involved in the Development of the National Model Code
Figure 5 depicts, at the federal government level, the key actors involved in the code development system, their relationship as well as their shared roles and responsibilities. The three main actors shown in the figure are the Canadian Commission on Building and Fire Codes (CCBFC), the National Research Council (NRC) and the national / international standard organizations. The figure contains three large boxes for each of these actors.
The box on the left side of the figure shows the CCBFC. The CCBFC includes 11 standing committees and is responsible for developing and approving the NMC before publication by NRC. Within the CCBFC box is included another box, representing the Provincial/Territorial Policy Advisory Committee on Codes (PTPACC), which includes Building Regulatory Officials from P/Ts who provide policy advice to the CCBFC. The box illustrates the importance of this committee for the governance of the CCBFC. More details on the CCBFC and PTPACC are provided in the main document.
The box on the right side of the figure shows the NRC. This box contains three smaller boxes. Each of these smaller boxes presents the three areas of NRC that play a role in supporting the CCBFC. These smaller boxes are: NRC-Canadian Code Center (NRC-CCC), NRC-IRC / Construction Portfolio Research programs, and Canadian Construction Material Center (NRC-CCMC). The roles of each NRC function are described in the main evaluation report. The arrows that link the three smaller boxes show that while the NRC-IRC / Construction Portfolio and the NRC-CCMC are supporting each other, both boxes also provide support to NRC-CCC.
To illustrate the supporting role played by NRC in support of the CCBFC (and vice versa), a double-sided arrow links these two boxes. More specifically, the arrow shows that NRC-CCC addresses requests for technical information which influence NRC's research activities. In turn, NRC influences the agenda of the CCBFC by stressing key issues that need to be examined by the CCBFC committees.
The third, somewhat smaller box in the middle of the figure shows the role of national / international standard organizations in Canada, given the fact that the NMC sometime references standards maintained by these organizations.
The membership of the Commission and the technical committees is balanced to ensure that all relevant sectors and geographical areas of the country are represented in this organization.Footnote 14 As such, the expertise of the industry, the regulatory community and various interest groups contribute to the development of codes in Canada. The CCBFC is also supported by the Provincial/Territorial Policy Advisory Committee on Codes (PTPACC), a committee made up of senior building regulatory officials appointed by the P/Ts. The main role of the PTPACC is to provide policy advice to the Commission to ensure that the content of the code reflects the needs and priorities of their stakeholders.
For its part, NRC-IRC's primary functions are to provide administrative and technical support to the CCBFC through the NRC Canadian Code Center (IRC-CCC). IRC-CCC coordinates the in-person/videoconference meetings of the members (including preparing the meeting's agenda, minutes and the committee workplans) as well as the translation of documents, the publication of the codes and the communication/marketing functions surrounding code sales to the public. IRC-CCC staff also ensures that the committees have access to R&D competencies and facilities to support the decision-making process surrounding code development. When technical information is unavailable, IRC-CCC staff can bridge the gap and provide the CCBFC with access to the core research competencies at NRC-IRC. NRC-IRC will undertake, usually in collaboration with industry, a research project to obtain the information needed.
According to the Inter-Jurisdictional Regulatory Collaboration Committee (IRCC, 2010; p. 28), "building regulations are legal instruments intended to ensure that buildings, when constructed in accordance with the regulations, provide socially acceptable levels of health, safety, welfare and amenity for building occupants and for the community in which the building is located".
As discussed in the previous paragraph, the research competencies and facilities of NRC-IRC strongly support the code development process by conducting scientific research to advance knowledge around the safety, health and sustainability of buildings. In addition to the research expertise provided by the Institute, NRC manages the Canadian Construction Material Center (IRC-CCMC). IRC-CCMC started in 1988 when CMHC's Material Acceptance Department moved to NRC after consultation with the P/Ts. In doing so, the Center became Canada's national technical evaluation service for innovative building products, which eliminated the delivery of similar services by the P/Ts. The goal of this Centre is to provide an evidence-based opinion on the compliance of innovative products/technologies/materials with the NMC (i.e., through evaluation and listing services) and/or the amended version of the code adopted by the P/Ts. These types of services, which exist in several countries, are used by the manufacturing industry to market their products and by code enforcement officials to assess the compliance of new products/materials with the code.
To have the force of law, the NMCs need to be adopted by the parliaments of the respective P/Ts. In some cases, the codes are amended to reflect regional preferences / needs and to address other specific issues that will be discussed later in the report. In most provinces and territories, the enforcement of the code is delegated to the municipalities, which have implemented inspection regimes that vary strongly from one region to the other.
2.5.2 Key Actors Involved in the Canadian Construction Innovation System
A basic model of the construction innovation system is shown in Figure 6.Footnote 15 The figure calls attention to three important points about the construction innovation system: 1) many stakeholders with varying roles and responsibilities are involved in the development and application of the building code; 2) each actor is interrelated with the others; and 3) several conditions and success factors need to be in place for such a system to function and/or to be efficient.
At the first decision-making level, NRC provides policy advisory services to the P/Ts in the form of the NMC. In the second decision-making level, P/Ts have the flexibility to adopt the code "as is", to adopt it with amendments, or not to adopt it. At the third level, the municipalities are responsible for the enforcement of the codes. In Canada, these enforcement regimes vary significantly from one region to the next. The fourth decision-making level includes industry and civil society, who participate in the code development process through the CCBFC and public consultations at the national and/or provincial levels. Also falling within this decision-making level is the responsibility of the building industry to apply the building codes in new construction.
Figure 6 not only illustrates the complexity of the program delivery model surrounding the NMC but also reveals that the achievement of code objectives (i.e., health, safety, accessibility and sustainability) is explained by the actions of multiple stakeholders. Moreover, the figure demonstrates that specific factors and conditions need to be in place at different tiers of decision-making for code objectives to be achieved. For instance, the achievement of the NMC objectives depends on the adoption of the NMC by the provinces and territories, the enforcement of the building code by the municipal inspectors and by the proper application of the codes in new construction by builders.
Figure 6 also illustrates the central role played by NRC in the Canadian construction innovation system, through the provision of direct services to the provinces (i.e., development of the NMC through the CCBFC and IRC-CCC, R&D programs and services), the municipalities (i.e., IRC-CCMC evaluation reports) and the construction industry (i.e., all R&D programs and services).
Figure 6. Overview of Canada's Construction Innovation System
Long description of Overview of Canada's Construction Innovation System
The information presented in this figure is organized using a matrix with four quadrants. The column headings, at the top of the image are (from left to right) Governors/Regulators and Regulatees. From a horizontal perspective, the top row is labelled Policy Setting / Code development, while the bottom row is labelled Policy / Code Implementation.
The figure includes five boxes that represent each of the five key actors involved in the construction innovation system. These actors are defined as "Tiers", based on their level of authority in the decision-making process related to code development.
The locations of the boxes within the four quadrants illustrate the respective roles of each actor in the code development process. The role of each actor is determined by two main functions: their role in the governance of the system (i.e., Governors/Regulators or Regulatees) and their role in the administration of the system (i.e., Policy Setting / Code Development or Policy / Code Implementation). The following table presents each actor as along with their specific function within the system. It also identifies the locations of the different boxes (i.e., actors) in the figure.
|Tier level||Location in matrix||Key Actor||Responsibilities in the innovation system||Role in the governance of the code system||Role in the administration of the code system|
|Tier 1||In the middle of the figure, touching all four quadrants, but with a greater presence in the upper left-hand quadrant.||NRC and the CCBFC||See description presented in Figure 5||Primarily Governors / Regulators||Development and Implementation of the code (because NRC sometime provide advice to provinces or municipalities regarding the interpretation of codes|
|Tier 2||Upper left quadrant||Provinces and Territories||
||Governors / Regulators||Policy Setting / Code Development|
|Tier 3||Lower left quadrant||Building Inspection and Enforcement Agencies (e.g., municipalities and building inspectors)||Ensure compliance of buildings with codes through review of plans and inspections depending on specific P/T enforcement system||Governors / Regulators||Policy / Code Implementation|
|Tier 4||Upper right quadrant||Key Market Place Intermediaries (e.g., industry associations, consumer groups, municipality associations, bank/mortgage lenders, insurance companies)||
||Regulatees||Policy Setting / Code Development|
|Tier 4||Lower right quadrant||Construction and Related Industries (e.g., manufacturing of construction products & materials)||
||Regulatees||Policy / Code Implementation|
Note: A tier decision making level can include more than one key actor, as shown in the case of tier-4.
The relevance of NRC-IRC was assessed by examining the needs of the Institute's stakeholders, and the extent to which NRC-IRC activities are aligned to the current and future needs of the stakeholders. Additionally, the evaluation assessed the extent to which the activities of NRC-IRC were consistent with federal roles and responsibilities, and aligned with the federal government properties and NRC's current strategic outcomes.
3.1 Need for NRC-IRC
- What are the needs of the NRC-IRC stakeholders?
- To what extent are the activities of NRC-IRC aligned to current and future needs of the Construction Portfolio's stakeholders?
- NRC-IRC has established the proper strategies and mechanisms to identify and prioritize the needs of its stakeholders in specific niche markets. Many other needs remain to be addressed in other research areas.
- Although NRC-IRC has addressed the needs of its direct clients and stakeholders from the private sector, a few federal government departments expressed uncertainties around the Institute's ability (as a new Portfolio) to address their current and future needs given communication challenges around the new strategic orientation of NRC.
- NRC-IRC has been successful at establishing/managing the key components associated with the code development process and, in collaboration with the CCBFC, has developed a version of the code that addresses the needs of the P/Ts.
This section addresses the alignment of NRC's activities with the needs of its clients and stakeholders. It presents the main characteristics of the Canadian construction sector in so far as they affect NRC-IRC's role in support of the sector. This is followed by a description of the mechanisms used by NRC-IRC to identify and prioritize the needs of its stakeholders. The last two sections present, for the R&D and code-related needs of clients / stakeholders respectively, what these needs are and the extent to which these needs were addressed by NRC-IRC.
3.1.1 Characteristics of the Construction Sector that Influence NRC-IRC's Ability to Meet Industry Need
The evaluation identified five characteristics of the construction sector that affect NRC-IRC's ability to meet the needs of the construction sector. These include the large size and diversity of the value-chain; the importance of the sector to the Canadian economy; the fragmentation of the sector; the low investment in R&D; and the risk-averse nature of the sector.
Large Size and Diversity of the Construction Value-chain
The construction sector includes a mosaic of players, involved in three broad segments: residential buildings; commercial and institutional buildings; and public infrastructure. Each of these segments includes a large diversity of companies and associations with various needs and priorities. Some of the key players include:
- Construction material industry (e.g., wood, steel, concrete);
- Manufacturers of construction products and technologies, including prefabricated houses, ventilation systems, windows/doors manufacturers;
- Architects and engineers;
- Testing laboratories;
- Builders and contractors;
- Civil engineering industry;
- The owners and operators of large building parks and/or infrastructure such as bridges and highway bridges (including federal and provincial governments);
- Residential building owners;
- Regulators (Federal, provincial and municipal governments);
- Standard organizations (e.g., Standard Council of Canada); and,
- Financial institutions and insurance companies.
Importance of the Industry on the Canadian Economy
Evidence from the literature review as well as interviews with internal and external stakeholders revealed that the construction sector has a significant impact on the Canadian economy. The construction sectors contribution to the Gross Domestic Product (GDP) for the Good-Producing sector, which reflects the wealth creation in the country, was $76.5 billion in 2011, second only to manufacturing (see Table 2).Footnote 16
|Sector||Contribution to Canada's GDP (Value; in Millions)||Contribution to Canada's GDP (Proportion of total GDP|
|Mining and Oil and Gas Extraction||57,443||4.0%|
|Agriculture, Forestry, Fishing and Hunting||29,093||2.3%|
Source: Statistics Canada, Canada Industry Statistics, GDP Canadian Economy (NAICS 11-91); retrieved from http://www.ic.gc.ca/eic/site/cis-sic.nsf/eng/h_00013.html on June 25, 2013.
Note: The construction sector described in this table does not include manufacturers of construction-related products and operators of highways, streets and bridges. This is a segment with which NRC-IRC works and is a key contributor to innovation. The way in which sectors are traditionally defined according to Statistics Canada NAICS Codes does not allow for the amalgamation of information on the construction industry and manufacturers of construction related products.
Further evidence of the importance of the construction sector to the Canadian economy is demonstrated by the fact that in 2011, the construction industry employed approximately 1.2 million workers, representing approximately 33 percent of the total employment in the goods-producing sector in Canada.Footnote 17 By comparison, 1.7 million Canadians worked in the manufacturing sector and 300 thousand in the agriculture sector.Footnote 18
According to the Construction Portfolio planning documents, the value of Canada's current existing infrastructure and building assets is worth more than $5 trillion. In addition to this, the built environment represents a notable proportion of Canadian citizens' assets (e.g., their homes). In 2005, the principal residence and other real estate of Canadians represented just over 40 percent of their total wealth.Footnote 19 The importance of this specific asset for Canadians highlights the importance of ensuring the sustainability and durability of buildings in Canada.
Finally, from a public infrastructure perspective, the existence of reliable and efficient roads, water and wastewater infrastructure systems are key conditions for economic activity and for the well-being of Canadians.
Fragmentation of the Industry
The construction industry is strongly fragmented into a large number of small companies. In fact, micro companies (between one to four employees) represent 60.4 percent of the industry while 38.7 percent of companies are small employers (between five and 99 employees).Footnote 20 The fragmentation of the industry explains, in large part, the low level of R&D capabilities and investments made in the construction sector, as described below.
Low R&D Investment
Very few resources are invested in R&D in the constructor sector. As shown in Table 3, the construction sector's level of R&D investment is lower than that of other goods-producing sectors. While most construction companies invest very limited amounts of resources in support of innovative projects and ideas, largely due to their small size, certain manufacturers and large developers in the construction sector do have significant resources and carry out significant amounts of research, development and innovation.Footnote 21
|Sectors||Value of R&D Expenditures (in millions)||Proportion of Total R&D Expenditures for all Industries|
|Mining and Oil and Gas Extraction||842||5.4%|
|Agriculture, Forestry, Fishing and Hunting||105||0.7%|
Source: Statistics Canada, Sciences Statistics, Industrial Research and Development 2007 to 2011, Catalogue No. 88-001-X, p. 12
Note: The construction sector described in this table does not include manufacturers of construction-related products and operators of highways, streets and bridges. This is a segment with which NRC-IRC works and is a key contributor to innovation. The way in which sectors are traditionally defined according to Statistics Canada NAICS Codes does not allow for the amalgamation of information on the construction industry and manufacturers of construction related products.
Industry and Consumer's Aversion to Risks
Another recurring theme observed in the literature and mentioned frequently by internal and external interviewees was the risk-averse nature of the construction sector, particularly when it comes to the adoption of innovative products and materials. This is explained by the significant liabilities and risks to health and safety that can result from the failure of building-related products and materials. For example, in the 1970's, urea formaldehyde foam insulation (UFI) was installed in houses with the support of the federal government, only later to be deemed a health hazard and banned in 1980's. Significant costs were incurred by all parties involved, demonstrating how the introduction of a new technology into the market can lead to liabilities. This resulted in a recognized need for a service, such as IRC-CCMC, in Canada.
The consumer's perspective also plays a role in the risk-averse culture characterizing the construction sector. As noted by an internal interviewee, a very limited number of Canadians are willing to have their houses used as test beds for innovative products given the potential risks to their health and safety. Moreover, as noted earlier, the purchase of a house is for most Canadians one of the most, if not the most, important economic transactions in their lives and thus any failure related to their investment could have a significant impact on their financial situation and/or well-being.
3.1.2 Strategies and Mechanisms to Identify and Prioritize Stakeholders Needs
A review of internal documents and findings from the case studies as well as interviews with program management and external stakeholders indicate that NRC-IRC uses a variety of mechanisms to identify the needs of its clients and stakeholders. NRC-IRC engages in on-going consultations with key players in the value chain to identify the current and future technological trends in the industry. NRC-IRC also draws on knowledge from national and international scientific literature as well as insights from the research community through the on-going interactions of NRC-IRC staff with peers and clients. For example, NRC-IRC's work in the area of high performing concrete began as a result of the Institute's foresight into trends and future research directions.
Other sources of information used by NRC-IRC to identity and prioritize the needs of its stakeholders include the technical committees of the CCBFC, the PTPACC, the IRC-CCMC standing committee on technical evaluation as well as other federal government departments. Findings from the case studies conducted as part of the evaluation indicate that, many times, NRC-IRC initiates large research consortia following the identification of a need by the construction sector through the CCBFC committees or following a request from federal government departments such as NRCan, CMHC or Health Canada. As an example, NRC-IRC was invited by NRCan to assess the performance (i.e., energy consumption) of green buildings and to work with them to develop the Energy Code 2011.
The evaluation also identified a series of challenges faced by NRC-IRC with regards to its ability to identify and prioritize the needs of its stakeholders. First, the large number of players and the diversity of their needs have been found to be a challenge given the limited resources available to the Institute. The necessity for NRC-IRC to remain neutral has been identified as a second challenge. In fact, as a government organization, NRC is expected to support all of the stakeholders and not favor one segment over the other. This means that research conducted by NRC-IRC must balance the needs and concerns of all stakeholders, which is particularly important to maintain the perception of neutrality.
3.1.3 R&D Needs of NRC-IRC Stakeholders
The first sub-section describes the R&D needs of NRC-IRC's clients and stakeholders. It is followed by a discussion on the extent to which NRC-IRC met these needs.
Description of R&D Needs of NRC-IRC Stakeholders
One of the principal needs of the private sector identified as part of the literature review and interviews with external and internal stakeholders is the provision of neutral, third-party assessment services of the performance or the compliance of products/materials with building codes and standards. The need for third-party assessment of performance/compliance stems from companies' needs to access the Canadian market as well as foreign markets. Other barriers to market access include the absence of standards for product use or integration into complex building systems / environments.
Federal government departments also have specific R&D needs that are required for the achievement of their mandate. For example, the Department of National Defense (DND), Health Canada (HC), Transport Canada (TC), Canada Mortgage and Housing Corporation (CMHC) and Natural Resources Canada (NRCan) have significant R&D and technical needs in support of their mandates. These mandates include the health of Canadians, ensuring the durability of buildings, energy conservation and sustainability of resources in Canada.
Federal, provincial and municipal government needs as they relate to large public infrastructure (e.g., bridges, roads, water and wastewater mains) must also be considered. As owners and operators of large scale infrastructure, government departments are required to maintain this infrastructure in good condition to ensure the quality of life of Canadians. Over the last 20 years, the condition of Canadian infrastructure (i.e., all levels of government included) has deteriorated, resulting in a deficit estimated at approximately $123 billion in 2007.Footnote 22 This finding, in conjunction with the limited financial resources of all levels of government in Canada, indicates that there is a demonstrable need for cost-efficient solutions and technologies for the management, retrofit and construction of large scale infrastructure.
Extent to Which the Needs of NRC-IRC Stakeholders Have Been Met
One of the main observations that stems from the evaluation pertains to the fact that NRC-IRC has addressed a large diversity of client needs in various research areas. This finding indicates a very broad strategic research orientation across many niche markets for NRC-IRC during the period covered by the evaluation. This appears to have changed as part of the new NRC strategy. More specifically, the organization seems to have narrowed its focus on initially four program priorities that are expected to evolve along with industrial and government needs. This is in alignment with the new NRC program-based management model where emphasis is on focused intended outcomes with a critical mass aligned to achieve major impacts for the Canadian economy.Footnote 23
That said, almost all of the external clients / stakeholders from the private sector and industry associations that were consulted as part of the evaluation felt that the programs and services of NRC-IRC met their needs. More specifically, NRC-IRC was strongly commended by private sector companies for the testing and validation services it provides. Services such as testing and evaluation, the CCHT laboratory, and IRC-CCMC were found to address a real need of the industry in terms of third party assessment. External interviewees consulted as part of the case study on CCHT mentioned that there is the potential for the facility to quickly become outdated due to aging technology. As such, there is a risk regarding the Construction Portfolio's ability to meet its clients' needs in this area
A few clients / stakeholders commented specifically on the fact that many projects proposed by NRC-IRC were well aligned with real problems faced by their company or sector. In this regard, one interviewee mentioned that "they [NRC-IRC] have a good crystal ball" and that NRC-IRC is not "trying to shove something down the throat of the industry." Despite this, there is some evidence that additional areas exist where NRC-IRC could offer services to meet industry R&D needs. These areas, identified in a report submitted by the Canadian Construction Association (CCA) as part of the Review of Federal Support to R&D, represent potential business opportunities for the Construction Portfolio going forward.Footnote 24
Moreover, findings from the 2010 IRC-CCMC Business Review indicate that NRC-IRC is meeting the private sector's needs as they relate to product evaluation / compliance through IRC-CCMC. The services offered by IRC-CCMC provide manufacturers with better access to the Canadian market and make it more likely that their product/system will be selected by architects, engineers or construction firms. In fact, the existence of IRC-CCMC is said to provide an even playing field for smaller manufacturers. Building officials and regulators also rely on CCMC's evaluation services for product approval as not all building officials have the same amount of knowledge and / or resources to interpret the information submitted by the proponent of an innovative product. IRC-CCMC is viewed as a reliable source of information whose services help to expose building officials / regulators to less risk.
The extent to which government departments felt that NRC-IRC met their needs over the last five years was mixed. While some departments expressed their satisfaction, a few indicated that communication with NRC-IRC had been challenging since NRC initiated its reorganization as part of its new strategy. More specifically, they cited a lack of responsiveness of the organization as well as the lack of clarity with regards to the new orientation of NRC and the Portfolio during the transition period. As a result of this, these individuals mentioned that although NRC-IRC was able to meet their needs a few years ago, they could not determine if the Portfolio could meet their current or future needs, going forward.
Recommendation 1: NRC should take the appropriate means to ensure that other government departments are aware and understand the new strategic direction of the organization and how the new NRC Programs and services can support their mandate.
3.1.4 Code-Related Needs of NRC-IRC Stakeholders
This section describes client / stakeholder needs related to building codes and also discusses the extent to which NRC-IRC has met these needs.
Description of Code-Related Needs of NRC-IRC Stakeholders
The literature review and interviews with external stakeholders revealed that the existence of the National Model Codes (NMC) serves many purposes, depending on the perspectives of the various stakeholders. For the provinces and territories, the NMC represents the main input for achieving their mandate of ensuring the health and safety of buildings located in their jurisdiction. For the contractors, the code helps them construct a building that meets the minimum health and safety requirements of the province or territory in which they operate. From the manufacturer's perspective, the performance requirements embedded in the code help them to determine the level of performance required for their product to comply with the building code and, as a result, facilitate market access for their products.
To increase their productivity and competitiveness, contractors and manufacturers also need to operate within a regulatory framework (i.e., the building code) that allows for the introduction of innovative ideas into the built environment. The literature review conducted as part of the evaluation showed that prescriptive building regulations may create barriers to market access for specific products or technologies. As noted by CCA in their report to the Federal R&D review panel, some of the specific actions that the government can make in this regard include the provision of: "an innovation-friendly regulatory regime, including increased emphasis on performance-based regulation".Footnote 25
The evaluation also found that several organizations (e.g., CCBFC, PTPACC), resources (e.g., IRC-CCC) and services (e.g., IRC-CCMC evaluation services) need to be in place for any code system to function properly. As one interviewee noted, "the code is organic and needs a whole system to be alive". This quote stresses the fact that for the code to meet the needs of all the stakeholders, a support infrastructure is required. More specifically, the establishment of a governance structure such as the CCBFC, services related to the publication, marketing and sales of the code as well as the existence of evaluation services like IRC-CCMC need to be in place for the system to function. The establishment of strong R&D competencies that provide scientific evidence in support of code development is another essential component of the code ecosystem. Evidence from the document review and interviews with external stakeholders also point to the need for neutrality with regards to the governance and administration of the code development system. The need for neutrality explains why the code development system, in most countries, is administered by a government organization.
Interviews with internal and external participants highlighted that for most provinces, there is a need to use the codes as a tool or mechanism to achieve specific policy goals related to issues such as climate change, building accessibility, or sustainability of natural resources. These needs have been found to vary significantly from one province to the other. While some provinces prioritize the energy efficiency of buildings, others were found to prioritize, for example, increased use of wood in mid-rise buildings.
Finally, the documents review identified that from the Canadian public's perspective, there is a need to have confidence that the buildings they live and work in provide an acceptable minimum level of safety and health.
Extent to Which Code-Related Needs of Stakeholders Have Been Met
The evaluation found that all of the elements required for the code system to function and address stakeholder needs are in place in Canada. In fact, with the exception of the services offered by IRC-CCMC, which will be discussed later in the report, very few issues were raised about the CCBFC, NRC-IRC, IRC-CCC or any elements that compose the code development system. Also, the evidence shows an increasing trend in the adoption of the NMC across jurisdictions in Canada. This latter element will be further discussed in the performance section of the report.
Moreover, almost all of the building regulatory officials from the provinces and territories consulted as part of the evaluation expressed a strong level of satisfaction with the 2010 version of the NMC.Footnote 26 Key representatives of the building industry were also consulted on their level of satisfaction with the NMC. These interviewees all expressed a strong level of satisfaction with the work done by NRC-IRC and the CCBFC in support of the code. The evaluation also found that both the industry and the P/Ts support the transition to objective-base codes and consider this transition to be, in the word of an interviewee, "a step in the right direction."
While almost all of the building regulatory officials consulted expressed a strong level of satisfaction with the 2010 version of the NMC, half noted opportunities for improvement with regards to the ability of the national process to address their emerging needs in various areas in a timely manner (e.g., mid-rise wood frame buildings, sustainability of natural resources, safety in residential care homes). In fact, this was the main reason provided by provincial representatives from Ontario when asked why the province maintains its own code development system. The perceived lack of responsiveness of the national system can be explained by the following elements:
- Diversity of stakeholder priorities: The national system operates within agreed upon policies and procedures, determined by the construction community as part of CCBFC (and not by NRC), to ensure that the NMC is consensus-based and neutral. However, many stakeholders, including politicians in P/Ts, have large stakes in the code content. Under these conditions, national priorities do not always reflect provincial priorities. For example, while British-Columbia placed priority on changing the height of mid-rise wood frame buildings in their building code (i.e., the number of stories allowed), this was not the case for other P/Ts. British Colombia placed priority on this issue given the importance of the wood industry to that province. Only when sufficient support was identified across other P/T's did this issue become a national priority for the NMC.
- Need for scientific evidence: Most code changes need to rely on sound scientific knowledge to ensure they meet the performance requirements embedded in the code and to address the concerns of all stakeholders affected by the change. Inevitably, the need for scientific knowledge creates delays in the code development process. The need for scientific knowledge is justified by the necessity to avoid any consequences on the health and safety of Canadians. In the case of mid-rise wood frame buildings, for example, there is a need to scientifically demonstrate that wood frame buildings meet fire performance requirements to ensure the safety of occupants and to manage the concerns of groups, such as fire safety officials.
- Need for resources to change the code: In order to make modifications to the code and conduct the required research, resources are needed. Such resources, however, are not always available because key partners are not willing or able to contribute to these R&D projects.
In light of these factors, it appears that the perceived lack of responsiveness of the national system to the needs of some of the P/Ts can be explained by a series of factors that NRC-IRC has little control over. The fact that the national system has been successful at establishing programs such as the mid-rise wood frame buildings or the energy code illustrates that the national system, within its margin of manoeuvre and fiscal ability, is responsive to the needs of the P/Ts.
3.2 Role of Federal Government
- To what extent are the activities of NRC-IRC consistent with federal roles and responsibilities?
- Is there a justifiable role for the federal government and, specifically, NRC in supporting the development of construction code?
- There is a role for federal government involvement in support of construction-related research given the need of various stakeholders to access strong multidisciplinary R&D competencies to address complex problems faced by the industry. For historic reasons, these competencies have been located within NRC.
- This is a role that NRC-IRC is best positioned to play due to its unique expertise and facilities, including: its multidisciplinary skills; its impartiality and credibility; its ability to bring together various players for multi-party and multi-disciplinary initiatives; and, its reputation for providing high quality and reliable data.
- NRC-IRC R&D activities and programs complemented, rather than duplicate, those of other federal or provincial government departments.
- NRC-IRC is best positioned to support the code development process. As a federal agency, NRC can ensure the harmonization of standards and regulations across jurisdictions and generate economies of scale by avoiding duplication of the code development process in each province and territory.
- The fact that NRC is a federal government Research and Technology Organization (RTO) provides credibility to the National Model Code (NMC) and facilitates the use of scientific evidence to support code change.
The extent to which the activities of NRC-IRC are consistent with federal roles and responsibilities was assessed by looking at the role of the federal government, and more specifically NRC-IRC, in construction-related R&D and in the national model building codes.
3.2.1 Role in R&D
The role of the federal government in construction-related R&D was assessed by exploring not only the role of the federal government, but also NRC's role on behalf of the federal government.
Federal Role in Construction-Related R&D
The role of the federal government in construction R&D stems primarily from the need for a neutral body, which possesses strong R&D competencies in a variety of construction-related areas, and which can provide leadership and expertise around complex and multidisciplinary R&D challenges facing the construction industry (e.g., the development of evaluation protocols to assess the code compliance of highly innovative products or the development of innovative concrete and its use in a real life context). For historic reasons, these competencies have been located within NRC and have never been developed to the same extent by the P/Ts or the private sector. The evaluation evidence shows that other government departments and private sector companies have relied on the unique expertise of NRC to address complex R&D challenges such as the fire safety of mid-rise wood frame buildings, the development of innovative concrete materials, or the evaluation of the performance of green-buildings. Moreover, the majority of interviewees from government departments noted that unlike the research conducted by NRC, research conducted by the private sector focuses exclusively on the needs of that particular company and not necessarily on the broader needs of the industry. Additionally, the results generated by the private sector are not often disseminated to the broader community. In contrast, the federal government was said to bring with it a broader, national perspective and to make results available for all.
The availability of affordable neutral third-party testing and evaluation services for the private sector also justifies the role of the federal government. In this regard, the unique skills and level of expertise that NRC possesses was found to provide a high level of credibility to the performance assessment of products and, consequently, help companies address barriers to market access. The national scope of the assessments provided by NRC, which always cover the requirements of both the NMC as well as the unique requirement of the codes in P/Ts, provides the main justification for federal involvement. In fact, in the 1990s, an agreement was established between the federal government (i.e., NRC and CMHC) and the P/Ts (with the exception of Ontario), which stipulates that only one evaluation service would be available to companies, in order to avoid duplication of services and additional costs for companies. Moreover, it is also interesting to note that evaluation services have always been the responsibility of the federal government as CMHC was previously responsible for services equivalent to those provided by NRC-CCMC.
Finally, a few external interviewees and findings from one of the case studies indicate that the federal government has a role to play in infrastructure-related R&D (e.g., research on high-performance concrete) due to its sizable investment in infrastructure (e.g., bridges), in order to ensure value for money of its investment.
Federal Role in Construction-Related R&D via NRC-IRC
The appropriateness of NRC-IRC conducting construction-related R&D was also investigated. Findings from key informant interviews, case studies, and the document review support NRC-IRC in this capacity. While a large majority of external stakeholders from client companies, industry associations, and government departments, said that they had other options to access similar services or research competencies (e.g., private sector or universities), the majority of them indicated that they chose to access services or collaborate with NRC-IRC because of the Institute's unique competencies and expertise. This finding was echoed in all of the case studies. Likewise, most external interviewees referenced the unique facilities, not readily available elsewhere in Canada and in some cases, North America, as the reason why they chose to work with NRC-IRC.
The 2010 IRC-CCMC Business and Governance Review confirms the appropriateness of IRC-CCMC within NRC, citing that as an independent, third-party with a public purpose, it is well positioned to find a balance between supporting innovation and managing risk.
NRC-IRC's ability to bring together various players for multi-party and multi-disciplinary initiatives was also found to be a reason for working with NRC-IRC, according to external stakeholders, findings from the case studies, and the document review. Likewise, NRC-IRC's ability to provide data that are of high quality and reliability, as well as its multidisciplinary skills to tackle complex issues (both within the Institute, and across NRC) were also identified as key factors that best position it to conduct construction R&D. As an illustration of the multidisciplinary nature of construction-related research, composites traditionally found in the aerospace sector have been used in the research and development of high-performing concrete used in infrastructure.
Of those external stakeholders that indicated that alternative options existed from which they could have received similar services or research competencies as those provided by NRC-IRC, the majority mentioned universities and private sector organizations. A few indicated that international organizations could provide the services they received from NRC-IRC without specifying why NRC is preferred as a service provider. However, other external stakeholders specifically discounted universities as a possible service provider due, in part, to the high turnover of student researchers and the fact that universities tend to publish their research results, whether they are positive or negative. For companies working to develop a new product, publication of all research results can be to their detriment.Footnote 27
In terms of duplication between the activities and programs of NRC-IRC and other federal or provincial government departments, almost all government stakeholders indicated that there is no duplication. Rather, some specifically highlighted the complementary role of the various federal partners involved in construction-related research. For example, NRCan CanmetENERGY conducts R&D on clean energy as it relates to buildings; however, it does so in support of policy work. Similarly, CMHC conducts housing research; however, the targeted clientele of their work is the builder and the home owner. Further evidence of the complementary as opposed to duplicative nature of the R&D conducted by NRC-IRC and NRCan CanmetENERGY and CMHC is found in the review of administrative and performance data, which indicates that both of these federal players have made use of NRC's unique competencies and facilities by having the Institute/Portfolio conduct R&D on their behalf.
In terms of the product evaluation services offered by NRC-IRC through IRC-CCMC, potential duplication exists only in so far as the province of Ontario offers a similar product evaluation service – Building Materials Evaluation Commission; all other provinces rely on the services provided by IRC-CCMC. That said, the province of Ontario has directed its Building Materials Evaluation Commission not to accept applications for new reviews unless IRC-CCMC confirms that it is not working on or about to work on an evaluation of that type of product or system.Footnote 28
Finally, NRC-IRC activities are in line with the mandate of NRC, as it relates to the following responsibilities included in the NRC Act: "undertaking, assisting or promoting scientific and industrial research in fields of importance to Canada", and "providing vital scientific and technological services to the research and industrial communities". Evidence from the evaluation confirms that NRC-IRC has been involved in multi-party collaborations that focus on research areas that impact most sectors of the economy and are of importance to Canada (e.g., energy efficiency; environment; health, safety and security; infrastructure). Likewise, the evaluation confirms that NRC-IRC also provided technical services to the construction sector (e.g., product testing; product evaluation services from IRC-CCMC).
3.2.2 Role of Federal Government in Support of Building Codes
In Canada, the safety of buildings and the health of its occupants is the constitutional responsibility of the provinces and territories. Nonetheless, the federal government has been involved in the development of building codes since the 1930s. In addition to this historical factor, the rational for NRC-IRC's involvement in codes stems from the fact that the agency is part of the central government of the country. This is similar to other countries (e.g., France, Germany, United Kingdom, European Union), where the central government plays a key role in the coordination and administration of the code system. As a federal agency, NRC-IRC is in a unique position to ensure the harmonization of regulations and to generate economies of scale for Canada by avoiding the duplication of 13 code development systems in each P/T. In addition to these two factors, the fact that a federal RTO is supporting the code development system was also found to be advantageous. These three points are further discussed below.
Harmonization of Standards and Regulations
Studies published by the Fraser Institute (Jones and Graf; 2001), the Canadian Institute of Plumbing and Heating (2012) as well as a publication (2010) by the Inter-jurisdictional Regulatory Collaboration Committee (IRCC) provide strong evidence of the negative impacts of transaction costs (i.e., compliance costs with building and other related regulations) on the competitiveness of any industry sector. Harmonization of regulations across jurisdictions (i.e., provincial, municipal levels) to reduce the compliance costs for the industry is therefore the cornerstone that supports the involvement of a central government in building regulations, a notion that was also confirmed by some building regulatory officials interviewed as part of the evaluation.
In Canada, the harmonization of standards and regulations is particularly important for the manufacturing industry, which sells its products across the country. This is less of an issue for contractors, since very few of them operate in more than one province.
When asked about the impact that would result from the absence of NRC's involvement in support of codes, many building regulatory officials indicated that they would have to consult and coordinate with the other P/Ts to replicate a centralized code system to avoid inefficiencies and to address the needs of their industry with respect to standardization of building regulations.
The importance of the role played by central governments in harmonizing standards and regulations is a phenomenon that is also observed in other countries. In fact, the evaluation also found that central governments in most countries (e.g., France, Germany, United Kingdom, European Union) examined as part of the literature review are playing a key role with regards to the coordination and administration of the code system. The only exception that was identified is the USA, where a not-for-profit organization is in charge of developing the building code. It is also interesting to note the there is a desire in the European community to accelerate the creation of a "Eurocode" in order to reduce technical barriers to trade within the European market.Footnote 29
Economies of Scale
The federal government's involvement in codes was found to have generated economies of scale for the country. The production of the National Model Code by the CCBFC and NRC-IRC allows the P/Ts to rely on these documents without having to fully pay individually for the cost of establishing their own code development system. In fact, almost all of the building regulatory officials consulted as part of the evaluation confirmed that their jurisdiction does not have the resources or the expertise to reproduce or lead such a complex process.
In the absence of a national code development system, a few external interviewees indicated that another option would be for them to adopt the USA code developed by International Code Council (ICC). However, the ICC code does not reflect the specific needs of Canadians stakeholders, and was therefore discounted by interviewees as a viable alternative.
Advantage of an RTO Leading the Code Development System
The evaluation also identified specific advantages of having NRC, an RTO, as the organization that spearheads the code development process. Findings from the literature review and interviews confirmed that the involvement of NRC-IRC in the administration of the code development process and in the research activities associated with this process provides a strong level of credibility and neutrality to the NMC. The industry's perception of NRC as a neutral, honest-broker was found to have facilitated IRC's ability to build research consortiums with key players that may not have collaborated under different circumstances.
In addition to NRC-IRC's neutrality, the co-location of NRC-IRC's research programs and the code development expertise (i.e., IRC-CCC) was highlighted as being advantageous. This co-location facilitated the use of scientific knowledge in support of code development. For example, research conducted by NRC-IRC on standards for mechanically attached roofs, wood frame buildings and on performance standards and tools to measure the sound insulation of wood frame buildings will inform the building code.
3.3 Alignment with Government and NRC Priorities
- To what extent are NRC Construction's activities aligned with federal government priorities and NRC's strategic outcomes?
- The activities of NRC Construction are consistent with current government priorities as they relate to: energy efficiency and climate change; major infrastructure; research and innovation for economic growth; and the health and safety of Canadians.
- NRC Construction activities address five of the six critical issues for Canada that NRC intends to affect under its new strategic direction, officially announced in 2013, including: economic development; pressures on natural resources; climate change; security; and a changing sense of communities.
One component of relevance is assessing the extent to which NRC-IRC's activities were aligned with the priorities of the federal government and NRC. However, the Institute's activities underwent a major transformation beginning in 2009-10 when NRC embarked on the development of a new strategy and organizational structure. The Institute became a Portfolio (i.e., NRC Construction) and new cross-NRC programs were developed which involve collaboration internal and external to NRC. While the new NRC Programs were approved in early 2013, outside of the scope of the evaluation, it was decided that the NRC Programs would be used to assess alignment with government and NRC priorities to ensure that the evaluation's findings were relevant and useful.
3.3.1 Alignment with Federal Priorities
Evaluative evidence gathered through the document review confirms that, in addition to supporting the mandates of its federal clients (e.g., NRCan, CMHC), the strategy, activities and objectives of NRC Construction are consistent with current government priorities as they relate to: energy efficiency and climate change; major infrastructure; research and innovation for economic growth; and the health and safety of Canadians.
- Energy efficiency and climate change: The Federal S&T strategy identifies four priority areas for enhanced investment and activity, including: environmental science and technologies; natural resources and energy; health and related life sciences and technologies; and information and communications technologies.Footnote 30 NRC's construction activities are strongly aligned with the natural resources and energy priority area (e.g., the National Model Energy Code; activities related to developing new technologies to reduce the energy costs in buildings).
NRC's construction activities related to energy efficiency are consistent with the federal government's commitment to climate change. In 2009, the federal government signed the Copenhagen Accord and committed to reducing its greenhouse gas (GHG) emissions to 17% below the 2005 level by the year 2020, in support of a low carbon economy.Footnote 31 Budget 2011 provided $148.8 million over five years (2011-2016) to continue and expand federal programs designed to improve our understanding of climate change. In 2007, the government also introduced ecoENERGY for Buildings and Houses, a program that ran until March 2012, and encouraged the construction and operation of more energy-efficient buildings and houses to reduce GHG emissions. Finally, the 2011 Budget announced the renewal of the Clean Air Agenda (CAA) with an investment of $870 million over two years. This funding will build on the considerable momentum gained from the investments in the CAA over the period of 2007-2011 which formed part of the Government's broader efforts to address the challenges of climate change and air pollution, with a view to ensuring a clean and healthy environment for all Canadians. Climate change is therefore a high priority for the current government, and is an area in which NRCs activities are well aligned.
- Major infrastructure: NRC Construction activities related to major infrastructure (e.g., concrete for infrastructure such as bridges) are also well aligned with the federal government's priorities, particularly the Government of Canada's Building Canada Plan. The federal government has supported over 43,000 infrastructure projects across Canada under the $33 billion Building Canada plan (launched in 2007), the stimulus phase of the Economic Action Plan (launched in Budget 2009), and subsequent investments. Economic Action Plan 2013 delivers a new Building Canada plan to build roads, bridges, subways, commuter rail and other public infrastructure, in cooperation with provinces, territories and municipalities.Footnote 32
- Research and innovation for economic growth: All of the activities of NRC Construction are consistent with the federal government's commitment to support research that contributes to the commercialization of new products, processes and services in support of economic growth, as outlined in the S&T Strategy, and further reinforced in Budget 2012. The Portfolio has helped, and will continue to help the private sector increase its sustainability by addressing their needs for increased competitive value of products and built assets (e.g., in the areas of wood and concrete structures; energy and water conversation; climate change mitigation; safety of built structures).
- Health and safety: Finally, NRC Construction work related to the National Building Code and the research activities in support of this area is aligned with the Federal Government's ongoing responsibility to protect the health and safety of its citizens.
3.3.2 Alignment with NRC Priorities
As part of its new organizational structure and strategic directions, officially announced in 2012-13, NRC intends to use its research and innovation programs to support the sustainable growth of Canada's business sector, reduce the risk in early-stage technology development and strengthen the ability of Canadian firms to address six areas of major public concern, including: economic development; pressures on natural resources; climate change; growing heath care pressures; increasingly complex security challenges; and a changing sense of communities.Footnote 33 NRC Construction activities address five of the six critical issues for Canada that NRC intends to affect. These include:
- Economic development: e.g., increasing the competitiveness of Canadian organizations (both public and private) through increased operational efficiencies (e.g., reduced energy consumption and/or costs); enhanced market access for innovative products in the Canadian and export market (e.g., performance evaluation of innovative products; improved appeal and uptake of wood products and systems); and national uniformity of the building construction regulatory environment.
- Pressures on natural resources: e.g., focusing on the issue of rising energy demand through R&D aimed at developing energy-saving technologies and improving the energy efficiency of buildings.
- Climate change: e.g., contributions to the reduction of GHG emissions through R&D aimed at energy conservation and enhanced energy efficiency in homes (e.g., energy efficacy building codes). Increased use of wood in the built environment is also expected to help reduced the GHG footprint of the construction industry since the use of concrete is one of the main sources of CO2 emission.Footnote 34
- Changing sense of communities: e.g., working to provide innovative solutions to the increasing pressures on infrastructure (e.g., reduce the costs of both the construction and rehabilitation of Canada's critical concrete infrastructure and extend its service life), enabling communities to produce more energy than they consume (e.g., building as energy producers); and supporting habitability in northern regions.
- Security: e.g., NRC has conducted research on mechanisms to reduce explosions on tankers (i.e., the BLEVE phenomenon) to improve the safety and security of Canada, and is now conducting research to ensure the security of critical infrastructure and the life of occupants in foreign embassies.
Going forward, the new business planning process in place at NRC requires that the work of NRC Construction, like all Portfolios, falls within the scope of the NRC Programs approved by NRC's Senior Executive Committee (SEC) following a rigorous stage-gate process, which covers the alignment of the proposed programs with the new strategic orientations of the organization and, more importantly, with the needs of the industry and Government. This process offers an additional safeguard to ensure the continued alignment of NRC Construction programs with NRC's priorities and approved strategic agenda.
Findings in this section focus on evaluation questions related to achievement of outcomes including: the Institute's reach; the extent to which the NMC has been adopted and is harmonized; the impact of objective-based codes on innovation; and the impact of NRC-IRC's R&D and technical services.
- To what extent and in what conditions has NRC-IRC been successful at reaching its targeted clients and collaborators?
- Between 2005-06 and 2011-12, NRC-IRC worked with 283 unique clientsFootnote 35 as part of 453 projects that generated $44,731,702. While the majority of these clients were large private sector firms, NRC-IRC clients from federal government departments accounted for a larger proportion of revenue generated by NRC-IRC.Footnote 36 In addition to this, NRC-IRC provided product evaluation services via IRC-CCMC to 795 unique clients.
- Despite a decrease (number of unique clients and projects) in NRC-IRC and IRC-CCMC's reach to new clients in the beginning of the period covered by the evaluation, the Institute's reach generally stabilized in the later years with only minimal fluctuations.
- The Institute's reach to innovative private sector firms has not been fully exploited, due in part to NRC-IRC's limited understanding of who the innovative firms in Canada were, where they were located and what their R&D needs were.
- In addition to the collaborative work the Institute engages in and the services it provides to clients, it also reaches external clients through the sale of the national model building code.
- While NRC-IRC has mechanisms in place that facilitate increased reach such as the use of consortiums to conduct R&D, working with Industry Associations and the Canadian Centre for Housing Technology (CCHT), challenges to CCHT's continued success in the future of facilitating reach were identified.
In order to better understand the outcomes that NRC-IRC has achieved, it is necessary to first assess the Institute's reach. Reach was explored by examining NRC-IRC's reach to external clients and collaborators (excluding those reached by IRC-CCMCFootnote 37), the reach of IRC-CCMC, and the reach of NRC-IRC's activities related to the national model building code.
4.1.1 NRC-IRC Reach to External Clients
In order to assess NRC-IRC's reach to external clients, the evaluation first considered client and project information captured in the Institute's Client Information System (CIS) database. This is followed by a discussion of the factors that influence client reach. Note that IRC-CCMC clients are excluded from this sub-section and are discussed in the subsequent sub-section.
Between 2005-06 and 2011-12, NRC-IRC had 283 unique clientsFootnote 38 (excluding those that received services from IRC-CCMC). NRC-IRC's reach to unique clients decreased in the early part of the time period covered by the evaluation, but was generally stable in the later years (with a few fluctuations; see Figure 7). Private sector clients constituted two thirds of NRC-IRC unique clients; the proportion increases to three quarters of the total number of NRC-IRC clients when industry associations are included. The federal, provincial and municipal levels of government represent, respectively, 8 percent, 7 percent and 2 percent of the total number of clients.
Figure 7. Number of NRC-IRC Unique Clients and Projects, and Total Revenue Generated by Fiscal Year Source: NRC-IRC Administrative Data
Long description of Number of NRC-IRC Unique Clients and Projects, and Total Revenue Generated by Fiscal Year Source: NRC-IRC Administrative Data
|Total number of unique projects||91||64||61||52||39||64||82|
|Number of unique clients||67||47||39||38||27||29||36|
The majority of NRC-IRC's unique clients were from Canada (i.e., 74% versus 24% from the USA and 2% from other countries). When looking at private sector companies only (i.e., small and medium-sized enterprises – SMEs- and large firms), NRC-IRC predominantly worked with firms in Ontario and Quebec over the course of the evaluation. Compared to the percentage of construction firms in each Canadian region, NRC-IRC worked with a disproportionately higher number of firms in Ontario and Quebec and a disproportionately lower number in the Atlantic region, the Prairies and British Columbia (see Error! Reference source not found.).
Figure 8. Proportion of Canadian SMEs and Large Firms in the Construction Sector by Province and Proportion of NRC-IRC Private Sector Clients (SMEs and Large Firms) by Province
Long description of Proportion of Canadian SMEs and Large Firms in the Construction Sector by Province and Proportion of NRC-IRC Private Sector Clients (SMEs and Large Firms) by Province
|Region||% of Canadian SMEs and large firms in construction sector||% of NRC-IRC clients (SMEs and large firms)||% of Canadian SMEs and large firms in manufacturing sector|
Source: NRC-IRC Administrative Data and Industry Canada Canadian Industry Statistics (2011)Footnote 39
As was expressed by an internal interviewee, NRC-IRC is working to expand its reach to private sector clients in British Columbia (e.g., working with British Columbia Hydro; collaborating with the Council of Forest Industry). While the Institute is aware of its limited reach to private sector clients in the Atlantic region, the low innovation capacity of this region was highlighted as a challenge for increased collaboration. Opportunities were also identified by the Institute to engage private sector clients from Alberta, which is characterized by a strong innovation capacity. Thus, NRC-IRC's reach to innovative private sector firms has not been fully exploited despite efforts to reach clients outside of Ontario and Quebec as part of the Cross-Canada series of Building Science Insights, where, in the opinion of NRC-IRC management, thousands of practitioners were provided information about NRC-IRC. As a national program, NRC-IRC should be in a position to collaborate or provide services where opportunities and needs exist.
Documents reviewed as part of the evaluation as well as interviews with internal stakeholders demonstrates that although the Institute appears to have a general understanding of the needs of specific market segments, this is not based on in-depth research or analysis. For example, the evaluation could not obtain clear information from the Institute on who the innovative firms in Canada were, where they were located and what their R&D needs were. As is discussed in the subsequent section, while NRC-IRC has had a high degree of interaction with private sector clients, the R&D investments of these clients has been limited, particularly when compared to that of government departments and associations. Given NRC's mandate to increase private sector R&D investment in Canada, there are opportunities for NRC-IRC to improve.
Recommendation 2: The new Construction Portfolio, should seek ways to increase its understanding of its potential targeted clients (i.e., innovative firms in the construction sector), including those outside of Ontario and Quebec.
NRC-IRC's reach to external clients was also explored by looking at the number of projects the Institute engaged in. NRC-IRC engages in technical service work as well as collaborative research projects. Between 2005-06 and 2011-12, NRC-IRC had 453 unique projectsFootnote 40 with public and private sector clients from Canada, and abroad (including the United States of America and other international locations). Of these projects, 90% involved only one client per project, while the remaining 10% involved multiple clients. Approximately 37% of the projects were classified as collaborative research with the remaining being fee-for-service work, including such services as full fee research, testing and consulting.
When looking at the involvement of clients in NRC-IRC projects, large private sector clients and SMEs were involved in 42 percent of the projects that the Institute worked on from 2005-06 to 2011-12 (see Figure 9). In contrast, federal departments that worked with the Institute during this period were involved in approximately 34 percent of the projects undertaken. That said, federal government accounted for the largest proportion of revenue generated by NRC-IRC within the seven years (i.e., just over 60%). These findings suggest that the federal governments departments, despite accounting for only a small proportion NRC-IRC's reach as it relates to the number of unique clients, do in fact play a large role in the Institute's overall level of activity.
Figure 9. Proportion of Projects that Clients were Involved in Between 2005-06 and 2011-12 and the Proportion of Revenue Generated
Long description of Proportion of Projects that Clients were Involved in Between 2005-06 and 2011-12 and the Proportion of Revenue Generated
|Client Type||Number of partner interactions||% projects clients invloved in||Total revenue from partner contribution||% of total revenue from partner contributions|
|Federal Government Departments||161||34.55%||$27,660,043.57||69.12%|
|Large private sector enterprises||112||24.03%||$2,848,150.84||7.12%|
|Municipal Government Departments||13||2.79%||$779,937.00||1.95%|
|Provincial Government Departments||34||7.30%||$451,561.01||1.13%|
|Public Non-Profit Associations||19||4.08%||$476,857.00||1.19%|
|Small and medium-sized enterprises||84||18.03%||$1,754,682.53||4.39%|
Source: NRC-IRC Administrative Data
As can be seen in Figure 7 (see page 30), the number of unique projects undertaken in each fiscal year decreased between 2005-06 and 2009-10, when it was at its lowest. What is noteworthy is the fact that the revenue generated by the Institute in 2009-10 increased substantially despite the decrease in the number of new projects. This appears to be largely attributable to an increase in revenues for the NRC-IRC Indoor Air Program from two projects with federal government departments (accounting for 84% of the Indoor Air Program revenue in 2009-10).
Factors Influencing NRC-IRC Reach
The evaluation identified various factors that influenced NRC-IRC's ability to reach external clients and collaborators. First, the general economic climate affected NRC-IRC's reach. The effects of the 2009-10 economic downturn are reflected in the lower number of new projects that the Institute started working in that fiscal year.
Similarly, the Canadian economy influences the budgets of other federal government departments, which are a key source of revenue for NRC-IRC. For example, in more recent years, CMHC has reduced its investment in housing-related research and NRCan has experienced budgetary cuts. This not only presents a risk to NRC-IRC's ability to support government department strategic objectives but could also have an impact on NRC Construction's ability to establish multi-partner collaborative research projects. As one external interviewee highlighted, the private sector often depends on the large contributions made by a federal government department to multi-partner research projects; in the absence of the contributions by public partners, the private sector partners would not be able to put forward sufficient funds to cover the associated research costs. In addition, the code cycle also affects the reach and level of activity of NRC-IRC. As one internal interviewee highlighted, at the end of a building code cycle, R&D related to the code is less prominent. Within the evaluation time period, 2009-10 corresponded to the end of a code cycle.
The evaluation also found that the Institute had several mechanisms in place that facilitated its ability to reach external clients. The use of consortia to conduct research was one strategy by which NRC-IRC worked to reach and included various players from the large value-chain that is characteristic of the construction sector in its projects. Likewise, by working with industry associations, NRC-IRC was able to reach many players from the construction sector (e.g., diffusion of research results) while only having to work with one partner. Unlike individual private sector companies, industry associations can access resources from outside sources, such as the government.
Finally, NRC-IRC's partnership in the Canadian Centre for Housing Technology (CCHT) facilitated increased reach to the private sector.Footnote 41 Since 2000, a range of 44 public and private sector partners have been involved in CCHT research projects, including gas companies, hydro companies, municipalities, universities, private sector organizations and federal government departments. Findings from the case study on CCHT conducted as part of the evaluation indicated that there are challenges to CCHT's continued success into the future, particularly as it relates to the relevance of the facilities. Specifically, interviewees from the case study noted that the twin houses were built in 1998 and while at the time these were viewed as state-of-the-art facilities, there is now a need to have these houses upgraded. Without the investment in upgraded facilities, the houses may become obsolete (e.g., the twin houses were built on R2000 energy efficiency standards, which NRCan is now working to update) and thus become ineffective and/or inadequate to support future technology needs.
Recommendation 3: Given that CCHT supports the needs of companies to assess the performance of their products and technologies, that it has contributed to NRC-IRC's ability to increase its reach to the private sector and that further investment will be required to update the facility, it is recommended that the Construction Portfolio assess the extent to which the facility is aligned with the new strategic objectives of the Portfolio, and if it is deemed a priority, work with CCHT partners to invest the appropriate resources to ensure CCHT's continued relevance to industry.
4.1.2 IRC-CCMC Reach
In order to assess IRC-CCMC's reach to external clients, both IRC-CCMC clients and projects were considered as well as factors influencing its reach.
Between 2005-06 and 2011-12, IRC-CCMC had a total of 795 unique clients. While IRC-CCMC's reach to unique clients decreased in the early years of the time period covered by the evaluation, it was generally stable in the later years (with a few fluctuations; see Figure 10). About half of IRC-CCMC clients were Canadian (close to 40% were from the USA and another 9% were international clients). This demonstrates the level of interest of manufacturers outside of Canada looking to bring their product to the Canadian marketplace, as well as the importance of IRC-CCMC as a strategy for market entry of new products and technologies.
Figure 10. Number of IRC-CCMC Unique Clients and Projects by Fiscal Year Source: NRC-IRC Administrative Data
Long description of Number of IRC-CCMC Unique Clients and Projects by Fiscal Year Source: NRC-IRC Administrative Data
Between 2005-06 and 2011-12, IRC-CCMC completed 1425 unique projects. While the number of unique projects decreased in the early part of the time period covered by the evaluation, it was generally stable in the later years (with a few fluctuations; see Figure 10, above).
The data available in the IRC-CCMC project database reflects two types of key products: listings and reports.Footnote 42 Of IRC-CCMC's projects, approximately 50% result in the production of a listing, and the other half result in a report. Of those projects designed to produce an evaluation report, the majority (93%) involve the evaluation of a product's compliance using pre-established protocol (e.g., those previously developed by IRC-CCMC for a similar product). This finding suggests that the majority of IRC-CCMC activities involve assessing compliance of innovative products to the code and that the development of evaluation protocols represents a small proportion of its activities. However, these services are unique to NRC-IRC and provide a strong value added to the industry in terms of surmounting barriers to market access for innovative products.
Factors Influencing IRC-CCMC Reach
IRC-CCMC's client reach is positively influenced by the fact that it is the only national service available in Canada.Footnote 43 As such, any Canadian or Foreign manufacturers that wishes to access listing or evaluation services is required to do business with IRC-CCMC.
Opportunities exist for IRC-CCMC to expand its services, and consequently its reach. The 2010 IRC-CCMC Business Review identified areas in which IRC-CCMC could expand its services, particularly as they relate to energy efficiency, water conservation, and sustainability/conservation.
4.1.3 NRC-IRC's Reach through the National Model Building Codes
An indication of NRC-IRC's reach from its activities in the area of the national model building codes is found by examining the number of national building code document purchases, as tracked in the Institute's administrative data. The total number of clients that purchased the 2005 National Model Building Code was approximately 11,000, representing 70,000 documents with a total dollar value of $10.5M. To date, the 2010 National Model Building Code has been purchased by close to 7,000 clients (the five year code cycle is not yet completed), representing 34,000 documents with a total dollar value of just over $7M. Likewise, to date, the 2011 National Energy Code for Buildings has been purchased by approximately 600 clients to date, representing just over 1,000 documents with a total dollar value of $267K. Thus, the sale of the building codes has contributed to the NRC-IRC's reach to external clients. NRC-IRC's reach with regards to code sales is influenced by the adoption of the NMC by the provinces and territories.
4.2 Adoption and Harmonization of Codes
- To what extent and in what conditions has NRC-IRC contributed to the harmonization of Construction Codes and accepted practices across Canada?
- While the 2010 version of the NMC is expected to be adopted by almost all P/Ts, two provinces have delayed adoption. The 2010 version of the code does not appear to address the needs of the provinces of Newfoundland and Ontario as these two provinces are not planning to adopt this version of the code.
- The existence of the NMC was found to be an efficient mechanism to ensure the harmonization of codes and standards in Canada.
This section examines the performance of the NMC by looking at the extent to which the code has been adopted by P/Ts as well as the level of code harmonization across the country. Given the multitude of players involved in the NMC development system (as described in Section 2.6.1), the evaluation focused on NRC-IRC's contribution to adoption and harmonization.
4.2.1 Code Adoption
The P/T's adoption of the NMC creates the market for NRC-IRC to sell the NMC and fund the national code development system. Given that code adoption delays have financial repercussions for NRC-IRC, ensuring an acceptable adoption rate is critical to the sustainability of the code development system. As is discussed in the subsequent two sections, adoption rates of the 2010 NMC vary by P/T and are influenced by various factors.
Adoption Levels of the NMC
Adoption rates of the 2010 NMC were found to be uneven across the country (See Appendix C for detailed information on adoption levels by P/Ts). While most provinces and territories adopted the three components of the 2010 NMC (i.e., the building, fire and plumbing codes) within the first two years of the code cycle, others have not adopted all three components. For instance, as of July 2013, Quebec had adopted the 2010 fire code, however, had not adopted the building and plumbing codes, while Alberta and Newfoundland had not adopted any of the three components of the 2010 building codes. That said, the legislative assemblies of Alberta and Quebec are examining bills to adopt all components of the 2010 NMC. Nunavut and PEI are also expected to have adopted all components of the 2010 NMC at some point in 2013-14. Newfoundland, however, while having adopted part of the 2010 building code, does not plan to adopt the fire code or the plumbing code.
Factors Affecting Code Adoption and/or Timeliness of Code Adoption
The literature review conducted as part of the evaluation identified key factors that facilitated or hindered the adoption of the NMC by the P/Ts. The factors listed below represent those that were deemed the most relevant based on the findings from the literature review and interviews with internal and external stakeholders. Factors that have a positive influence on the adoption level, as well as the timeliness of the code adoption, include:
- Early engagement of key stakeholders (i.e., the P/Ts) through the existence of a consultation body similar to the PTPACC;
- Neutrality of the code development system with limited political involvement (i.e., decisions are made by a third party organization, represented by a large diversity of stakeholders from the construction sector as opposed to being made by elected officials);
- Public consultations;
- Provisions in the provincial law for automatic adoption of the code as new versions are available (i.e., as is the case for Nova-Scotia and Manitoba); and,
- Buy-in and commitment of political decision-makers early in the code development cycle.
- With the exception of the last factor, in which there are opportunities for improvement, all the factors listed above are currently present in the code development system. Specifically, the governance structure of the national system (i.e., the CCBFC and PTPACC) as well as the Policies and Procedure of the CCBFC encompass all of these factors.
- Factors that hinder the adoption of the code by the P/Ts were also observed as part of the evaluation. These factors, which result in adoption delays rather than affecting the adoption of the NMC in itself, include the:
- Political nature (i.e., adoption, at the legislative level, of the code) of the code adoption process in certain P/Ts;
- Lack of coordination between the national and P/T public consultations; and,
- Perceived lack of responsiveness of the national system.Footnote 44
One of the key findings from the evaluation is that code adoption is strongly influenced by the P/T's political processes (i.e., legislative process). In fact, because codes are laws, the adoption of the code is required to follow the legislative process. Like all laws, code adoption can be delayed by provincial elections, which was the case in Quebec and Alberta. Nonetheless, a few key internal interviewees as well as the documentation consulted as part of the evaluation suggest that by increasing the engagement of political decision-makers at the beginning of each code development cycle, the effects of the political process could be alleviated.
Interviews with building regulatory officials also confirmed the negative impact of the lack of coordination between the P/Ts and national system with regards to public consultations. In fact, most interviewees identified that enhanced coordination between the national and P/T processes would decrease the adoption time. Interestingly, a few internal and external interviewees mentioned that efforts were made to improve this aspect of the code development process when the PTPACC was created but that it never materialized for a variety of reasons including the different code approval process of each P/T.
In light of the findings discussed in this section, and given the importance of timely code adoption to ensure the financial sustainability of the code development system, the following recommendation is proposed.
Recommendation 4: In order to accelerate the adoption of the National Model Codes by Provinces and Territories, it is recommended that the new Construction Portfolio:
- seek ways to increase the engagement of Provinces and Territories; and,
- take a leadership role in consulting with the CCBFC and PTPACC to coordinate/streamline the public consultation process.
4.2.2 Code Harmonization
The existence of the NMC was found to be an efficient mechanism to ensure the harmonization of codes and standards in Canada. The majority of external interviewees expressed a strong level of satisfaction with the current level of harmonization in Canada.
Although almost all of the building regulatory officials consulted as part of the evaluation felt that the harmonization of the code is critical for Canada, the same interviewees also noted the importance for the code system to be flexible to the specific needs of the P/Ts. In this regard, the evaluation found that with the exception of the territories, almost all provinces include amendments to the NMC before adopting it. In the words of one external interviewee, the complete harmonization of codes in Canada "is a lofty goal".
Two main types of amendments to the code were identified as part of the evaluation: 1) higher performance standards for specific code elements; or 2) code provisions that are not covered within the scope of the NMC. The first category was found to apply to historic practices or regional preferences in terms of what is perceived as "society's minimum acceptable level of risk". For example, some provinces may require the inclusion of sprinklers in specific type of buildings to ensure the safety of occupants while the NMC consultations concluded that this measure is not necessary at this point. Examples of the second type of amendment (i.e., addition of areas not covered by the NMC) include issues related to resources/energy conservation, as well as privacy or safety for specific building components in residential care occupancies.
Evidence from the literature as well as interviews with building regulatory officials suggests that the following factors have a negative impact on the level of harmonization:
- Policy priorities of certain provinces in certain areas (e.g., British Columbia made amendments to its building code to increase the height of mid-rise wood frame buildings);
- Sharing of roles and responsibilities for the code within a P/T (e.g., in Quebec, building safety against fire is the shared responsibility of the provincial government and municipalities. While the minimum level of safety in Quebec is determined by the fire code, municipalities can implement more stringent standards resulting in variance among municipalities);
- Code adoption delays (e.g., two provinces can operate, for a specific period of time, under two different versions of the NMC. Newfoundland, for instance, has not adopted all components of the 2010 NMC); and,
- The existence of the Ontario code (e.g., Ontario has its own code development process and code, separate from that of the NMC).
Despite the inherent challenges in achieving complete harmonization, interviewees from the private sector expressed a strong level of appreciation and support for the harmonization of codes.
4.3 Impact of Objective-based Codes on Innovation
- To what extent and under what conditions has the use/adoption of objective-based construction codes increased the use of innovative design or the adoption of innovative material and technology in the Canadian built environment?
- While instances of increased innovation were identified, objective-based codes have not yet resulted in widespread adoption of innovative products and design in the built environment. Despite this, early benefits of this new regulatory framework have been identified.
- The evaluation examined the extent to which the transition to objective-based codes (OBC) has resulted in increased innovation in the construction industry as well as what conditions and factors need to be in place for such a regulatory system to achieve its expected outcomes. This section discusses the context within which OBC emerged in Canada as well as the extent to which OBC have resulted in increased innovations in the built environment.
4.3.1 Background on Objective-based Codes
This section describes OBC by outlining their key characteristics as well as the main assumptions that underpin this regulatory approach.
Prescriptive Building Codes: A Perceived Barrier to Innovation
In 2005, the CCBFC published the first objective-based version of the national model codes. This new approach to building regulations marked a major change to the regulatory construction system in Canada.
An objective-based approach to regulations provides specific objectives that are mainly qualitative in nature and linked to the requirements embodied in acceptable solutions so that alternative solutions can be assessed for suitability against the acceptable solutions [IRCC; p. 29]
Before 2005, Canada's regulatory approach to building codes was prescriptive in nature. This means that the performance levels that needed to be achieved in the built environment were "embodied in a large collection of prescriptive specifications that dictated how a building must be built, including what materials may be used, [and] how they may be used" (IRCC, 2010; p. 28). Although this approach was successful in ensuring a desired level of health and safety, limited information was provided to the user about the rationale for these requirements. Moreover, the prescriptive nature of the code, which relied on years of construction practices and research, represented a significant challenge for the use of products / technologies / materials that were not directly referenced in the code. In fact, a literature review commissioned by NRC-IRC in 2001 identified the prescriptive nature of codes as a significant barrier to the adoption of innovation in the built environment.Footnote 45
This situation was not unique to Canada and, according to the literature review conducted as part of the evaluation, many other countries felt that traditional approaches to building regulations were inflexible and inhibited the adoption of new technology. As such, most countries sought new means to clarify the intent of regulations, reduce regulatory burden and encourage innovation without compromising minimal expected levels of performance. The result has been a transition by several countries to various forms of performance-based building regulations (i.e. codes based on defined performance outcomes with limited to no reference to perspective elements).
Key Characteristics of Objective-based Codes
Although Canada was late to initiate the transition to performance-based codes when compared to other countries (some did so as early as the late 1970s while the CCBFC initiated the transition around 1995), this delay enabled the CCBFC and NRC-IRC to draw on lessons learned from the experiences of other countries. The Canadian objective-based codes were developed by identifying clear objectives and performance requirements for all code provisions. This work was done by the CCBFC committees over a ten year period.
One of the unique characteristics of OBC is that it maintains prescriptive elements, referenced as acceptable solutions, to guide code users on how the code objectives and requirements could be met. This is different than performance-based codes, where all reference to prescriptive elements is usually removed. By having clearly defined objectives and performance requirements, an essential component of objective-based codes, building designers, builders and manufacturers are able to propose innovative products and materials to meet the code requirements. These, referred to as alternative solutions, are usually unique and approved on a case by case basis by building officials.
The literature review conducted as part of the current evaluation identified various key assumptions regarding the expected outcomes of objective-based codes when compared to prescriptive-based codes. These include:
- Improved clarity of the regulatory intent;
- Increased effectiveness in reaching regulatory objectives via alternative solutions;
- Encouraged and increased innovation without compromising performance;
- Use of novel technologies for greater levels of health and safety;
- Minimized compliance cost for industry; and,
- Reduced regulatory burden and rigidity for regulators.
The following section examines the extent to which the first four key assumptions were realized in the Canadian context.
4.3.2 Impacts of Objective-based Codes on Adoption of Innovative Technologies and Designs
With the exception of three external interviewees (i.e., two building regulatory officials and one private sector interviewee), there is a consensus among evaluation participants as well as the literature reviewed that OBC have not resulted in a significant increase to the adoption of innovative products and design in the built environment. In fact, most internal and external interviewees agreed that builders and building officials rely, for the most part, on the prescriptive requirements of the code to ensure compliance with the code objectives. As these interviewees noted, it is too early for significant changes to be observed and many conditions are necessary for OBC to fully achieve their expected results.Footnote 46 Nonetheless, a few external interviewees from private sector companies with strong innovation capabilities provided anecdotal evidence that OBC have facilitated the adoption of innovative designs and technologies for them.
Although limited evidence of increased adoption of innovation was found, some early benefits of this new regulatory framework were identified by internal and external interviewees. These early benefits include:
- Clarification of the code intent by having clear objectives and performance requirements: In fact, most external interviewees noted that as opposed to prescriptive-based codes, OBC provides a clear path and flexibility when an alternative solution is considered by the industry. As noted by an external interviewee: "the objectives make explicit what is otherwise implicit".
- Creation of the conditions to facilitate NRC Construction programs, such as the mid-rise wood frame building program: In the opinion of an external interviewee, the adoption of OBC has facilitated the establishment of the mid-rise wood frame building program at NRC Construction by allowing for more flexibility to modify the content of the code.Footnote 47
- Increased market opportunities for manufacturers: A few interviewees from the private sector mentioned that the adoption of OBC helped open the market for their innovative products and technologies and, consequently, helped them expand their business opportunities. This, however, was often conditional to having an IRC-CCMC approved product or the approval of an engineer or architect.
Overall, almost all interviewees were of the opinion that it is too early to attribute any change in innovative activities to the adoption of OBC. One interviewee indicated that it is a long process given the risk-averse culture of the industry. The same interviewee also stressed that it took ten years to convert the code into OBC and that it will take another ten before the industry begins to capitalize on the opportunities afforded by the OBC (i.e., the use of alternative solutions).
Two factors were largely found to explain why OBC have not yet resulted in increased innovation in the construction sector. These factors – training of building officials and efficient product evaluation services – are discussed below.
- Training of building enforcement agents: Almost all of the internal and external interviewees noted that a lack of training of building officials was a main barrier to the adoption of innovative products and designs. As noted by one external interviewee, "building inspectors are not equipped to determine if alternative solutions are ok. Some won't even look at it". According to findings from the literature (Moyes, 2005; p. 4), the growing complexity of construction combined with increased liability has resulted in many, including building officials, backing away from alternative solutions. The same interviewees noted that appropriate training of building officials in municipalities could help to address this issue. Although NRC-IRC mentioned that it would be in a good position to provide such training, the Institute also noted that resources were not available based on the current level of funding it receives from the cost-sharing model for codes.
- Efficient evaluation services (i.e., IRC-CCMC): The availability of efficient evaluation services, such as those offered by IRC-CCMC, was found to be an important factor affecting the extent to which OBC facilitate increased innovation. By providing a neutral opinion on the performance of a specific product and/or building material, evaluation services help reduce the risks and liabilities for building officials and other actors involved in the use of innovative products and/or materials (e.g., product vendors, builders).
For evaluation services to result in the desired outcome (i.e., increased innovation), building officials must not only be aware of the services but must also use the information (e.g., product evaluations) to support decision making around the use of innovative products. Findings from the 2010 IRC-CCMC Business Review show that building enforcement agents do in fact have a strong level of awareness of IRC-CCMC services (i.e., 80%) and that a large proportion (i. e., 87%) use IRC-CCMC product evaluations to support decisions regarding the approval of products and materials. It can therefore be concluded that NRC-IRC has been successful, to a certain extent, at informing building officials about IRC-CCMC services and that these services are used. However, in order to fully support innovation, the provider of the evaluation services must be able to meet market demand. In this regard, external interviewees noted that there had been a backlog at IRC-CCMC, which slows down the introduction of innovative products and materials to the market. Institute records indicate that IRC-CCMC has successfully reduced this backlog in recent months. However, it does not appear that improvements were made to the time it takes for IRC-CCMC to provide its evaluation services, as is further discussed in Section 5.2. In other words, this means that although IRC-CCMC was successful at addressing the backlog of project, it does not mean that it was successful at achieving efficiency gain (i.e., timeliness of services, client satisfaction issues) with regards to the delivery of its services.
4.4 Impacts of NRC-IRC's R&D and Technical Services
- To what extent and in what conditions has NRC-IRC contributed to the commercialization and/or increased market access for Canadian products and technologies?
- To what extent and in what conditions have NRC-IRC's activities contributed to the health, safety and accessibility to building for Canadians?
- NRC-IRC has produced outputs that are highly diversified, reflecting the range of activities undertaken by the Institute in various research areas. Outputs included: knowledge creation, test results from compliance / performance assessments of products and materials, new test methods and protocols, industry guidelines and practical tools for industry, development of new/improved products and technologies, and tools in support of decision-making.
- Specific conditions and factors were identified as key determinants in the achievement of longer-term impacts for large scale R&D projects. These included the existence of appropriate knowledge and technology transfer mechanisms, the inclusion of key stakeholders in research consortia, code change and technology demonstration projects.
- There is anecdotal evidence to suggest that NRC-IRC R&D activities have, in some cases, increased market access and growth of the industry, resulted in cost-savings for public sector organizations and homeowners, improved fire safety, and contributed to the health and quality of life of Canadians. Environmental benefits resulting from NRC-IRC's R&D activities are also expected but have not yet been realized.
This section presents the impacts of NRC-IRC's R&D and technical service activities. Since the impacts of most publically funded R&D programs takes many years to come to fruition, impacts of NRC-IRC activities undertaken before 2005-06 (the beginning of the evaluation scope), were also taken into consideration. Before launching into a discussion of the Institute's impacts, the outputs are first discussed as well the factors and conditions required for the outputs to translate into impacts.
Findings from the case studies and interviews with clients show that NRC-IRC has produced high quality outputs for which external clients have expressed a strong level of satisfaction (this is further discussed in Section 5.2). These outputs are highly diversified, reflecting the range of activities undertaken by the Institute in various research areas. In fact, a large proportion of the outputs produced by NRC-IRC have targeted problems faced by segments of the construction industry as opposed to problems faced by single companies. Outputs delivered by NRC-IRC have included:Footnote 48
- knowledge creation;
- Test results from compliance / performance assessments of products and materials;
- New test methods and protocols;
- Industry guidelines and practical tools for industry;
- Development of new/improved products and technologies; and
- Tools in support of decision-making.
Some of these outputs were found to be transferred to industry and practitioners as part of direct services, R&D collaborations as well as through communication activities such as: the conduct of Business Science Insight seminars across Canada (attendance of over 3.500 practitioners), and the use of publications such as Construction Innovation Newsletters (24,000 subscribers), the publication of Construction technology updates and communication in trade magazines.
4.4.2 Factors Influencing the Occurrence of Impacts
Various factors were found to have influenced the extent to which the outputs produced by NRC-IRC resulted in impacts. These factors include:
- Existence of appropriate knowledge and technology transfer mechanisms;
- Integration of key stakeholders in research consortia;
- Codes change; and,
- Technology demonstration projects.
While not all of these factors need to be addressed as part of all projects, many were found to be critical for most large-scale projects undertaken by the NRC-IRC. In many cases, the presence (or absence) of these elements explained the extent to which outputs translated into impacts. In fact, projects that have generated the most significant impacts are those that have successfully addressed the factors relevant to their specific context. For example, NRC-IRC's self-curing high performance concrete project and sound insulation for wood frame building project have successfully addressed these factors to facilitate the occurrence of impacts. Each of the four factors is described in greater detail below.
Existence of Appropriate Knowledge and Technology Transfer Mechanisms
Various strategies and mechanisms have been used by NRC-IRC and its partner to diffuse the knowledge generated as part of research projects. In cases where the knowledge generated may be of interest to homeowners, builders, contractors or technology users, the publication of on-line practical guides/tools on NRC website or on the websites of government partners such as HC and CMHC are appropriate knowledge / technology transfer mechanisms (as evidenced by the number of downloads). In cases where the research community is the target audience, successful communication mechanisms have included the publication of papers or technical reports in journals as well as NRC-IRC researcher participation in scientific conferences.
Interviews with federal government departments identified a key risk with regards to the diffusion of knowledge in the form of practical guides. In the past, CMHC provided financial support to transform some of the technical material produced by NRC-IRC into a form that could be used by a non-technical audience. However, in FY 2009-10, CMHC's budget was reduced in two areas, fire safety and acoustics, limiting the extent to which they will be able to translate knowledge to facilitate its transfer. In fact, the absence of funding to help develop communication material related to leakage performance of window/wall interfaces has delayed the publication of the practical guide and explains, in part, why no impact was identified when the evaluation was conducted.Footnote 49
Inclusion of Key Stakeholders in Research Consortia
Having the appropriate partners (i.e., key stakeholders) involved in research consortia was found to play a large role in the achievement impacts. For example, in cases where building / infrastructure owners were the targeted beneficiaries, securing their involvement in the consortium was found to be critical to the success of the project. In fact, although building / infrastructure owners and administrators are not technology developers, these stakeholders play a key role in driving the technology direction of the construction sector. As such, their involvement was found to facilitate the acceptance of innovative solutions and technology. Case studies on the performance of green buildings, solid state lighting (SSL), and self-curing high performance concrete illustrated the importance of involving building / infrastructure owners. On the other hand, the case study on the performance of green buildings demonstrated that resistance on the part of building owners at the beginning of the project can create significant delays in establishing a research consortium.
When the knowledge created as part of a research project is translated into a code provision accepted by the CCBFC and that code is adopted by the P/Ts, it is almost certain (i.e., assuming that the code is enforced properly) that there will be a change to the building environment, and that impacts will take place over time. As such, the adoption of code provisions based on the outputs generated by NRC-IRC and its partners has a significant impact on the extent to which impacts occurred. In the case of the Special Interest Group for Dynamic Evaluation of Roofing Systems (SIGDERS) project, in which a standard was developed for testing the performance of dynamic roofs, the limited impacts can be attributed to the fact that the standard has not yet been included in the Canadian code despite the industry's interest, as reflected by its significant contribution to the project over the years. While some regions in the USA or Canada (e.g., Vancouver) have adopted the new roofing standard, the degree of adoption is minimal considering the duration of the project (more than 20 years) and the resources invested. It is expected, based on the evaluation findings, that the adoption of this standard in the 2015 version of the NMC should increase the impacts stemming from this project.
While inclusion in the code was found to play an important role in facilitating change, findings from the case studies and interviews with some external stakeholders indicate that consumer preferences (i.e., market demand) is also a strong driver for change in the industry. For example, the case study on sound insulation for wood frame buildings demonstrated that the tools for measuring the acoustic performance of wood frame buildings have been used extensively by the industry despite the absence of changes to the code. From the perspective of industry, this tool should not be used to increase the minimum level of acoustic performance required for wood framed buildings but should rather become a tool to increase the minimum standard for acoustic performance of buildings. This example illustrates the two different worldviews between some regulators and industry with regards to code content. While regulators tend to view the code as a mechanism to achieve policy goals, industry, who is more sensitive to the cost associated with code change, tend to prefer less restrictive performance requirements.
Technology Demonstration Projects
Technology demonstration was also found to be an important factor in reducing industry's resistance to change and acceptance of new technologies. Where the inclusion of early adopters in research consortia is the first step toward the large scale adoption of new technologies, technology demonstration projects help illustrate the benefits to a broader audience of potential users. This factor often operates closely with the inclusion of building / infrastructure owners in research consortia as these key partners provide the test beds for new technologies. The use of self-curing high performance concrete in the construction of the North Channel Bridge in Cornwall is an example of a technology demonstration project that has raised the interest of other infrastructure owners and therefore contributed to increased impacts.
The evaluation identified five main areas of impacts that resulted from NRC-IRC's activities related to R&D and technical services. These areas of impact, which range from more immediate impacts to longer term impacts are discussed in the subsequent sections, include:
- Increased market access for products and growth of the construction industry;
- Cost-saving for public sector clients and homeowners;
- Increased quality of life and health of Canadians;
- Increased safety and security of Canadians; and,
- Environmental benefits.
Increased Market Access and Growth of the Construction Industry
The testing services provided by NRC-IRC, as well as the industry guidelines and practical tools developed over the years, have had an impact on the commercialization of new/improved products. The evaluation also found anecdotal evidence of increased growth of companies and sectors (despite the challenges faced in measuring impacts – see Appendix B for evaluation limitations and challenges). Some examples of increased commercialization of products and industry growth include:
- Fire performance of cross-laminated timber (CLT): One organization, who worked with NRC-IRC on the fire resistance of cross-laminated timber, has had their innovative material adopted in the USA's new building code, and is expected to be directly referenced as an acceptable solution in the 2020 version of the Canadian building code. Results from this work are also being used by consultants, architects and engineers in Canada to demonstrate CLT as an acceptable solution to meet the objectives of the building code. Further expected impacts of the wide scale adoption of this technology in Canada and elsewhere will be faster and easier assembly of mid-rise buildings;
- Fire performance of new materials: One Canadian company worked with NRC-IRC on a project to test the fire resistance of an improved polymer that they had developed. As a result of the fire testing done by NRC-IRC they were able to win more contracts for their services all over the world.
- Acoustical design guide software: One external interviewee noted that he has managed $1 billion worth of construction work where the NRC-IRC research results in the area of acoustics were taken into account. Another interviewee indicated that his company's sales have increased by approximately 25% per year over the past several years, a large proportion of which can be attributed to NRC-IRC research results in the field of acoustics. This interviewee is also using the knowledge gained through the NRC-IRC project to inform his clients and thus help create demand for his company's product.
- Performance of windows: Work conducted at CCHT on the performance of windows helped expand the market for one company's windows as it demonstrated the windows' higher energy efficiency in a larger geographic area than originally anticipated. This also helped the company expand to some USA markets that it had not previously penetrated.
NRC-IRC has also helped the Canadian wood industry address technical barriers to trade with Korea and Japan as a result of the work done initially for the measurement of sound insulation in wood buildings in Canada. In fact, NRC-IRC developed a new test method to ensure product / material compliance with Korean and Japanese standards and codes in order to help reduce barriers to trade with these foreign markets. However, no evidence of increased sales for the wood industry on these markets has been identified as other barriers appear to prevent this from happening.
The support provided by NRC-IRC with respect to the commercialization of early stage technologies such as solid state lighting (SSL) and vacuum insulated panels (VIP) have been found to address critical barriers to market access. However, very few impacts on the growth of companies could be observed as a result of these projects. This observation can be explained by the fact that these technologies have a medium level of technology maturity and, thus, are not fully ready to be commercialized. That said, the case study on SSL did reveal that NRC-IRC's work in this area has contributed to the adoption of SSL in offices by developing novel functionality, exploring lighting quality issues related to its new capabilities, and demonstrating these features to a wide range of audiences. Likewise, the information from the SSL project has been used by the CCBFC technical committees in the development of the 2015 National Energy Code for Buildings. Standards related to SSL will likely take a few years to be developed. While these types of projects (i.e., SSL and VIP) were found to yield limited short-term impacts on the industry, it is expected that the adoption of such technologies (e.g., SSL and VIP) will have a significant impact on the energy consumption of buildings in Canada. While no impacts on specific firms could be identified to date, NRC-IRC support was found to be necessary over the long-term to support the eventual commercialization of these technologies.
Cost-Savings for Public Sector Clients and Homeowners
Findings from interviews with public sector clients and the case study on NRC-IRC's work on self-curing high performance concrete provide strong evidence of cost-savings resulting from the Institute's work. Interviewees from the public sector noted that they have benefited from infrastructure investment decision support tools and detection systems for water leakages in various types of pipes, as well as innovative concrete developed by the Institute. Below are two examples of these types of projects, and the resulting (or expected) cost-savings.
- One interviewee from a municipal government noted that as a result of the municipality's partnership with NRC-IRC, a methodology was developed, which they could use to evaluate the condition of their assets (e.g., water mains). As a result of the enhanced diagnostic methodology, the city reduced its annual spending on water main repairs from $15M to $7.5M.
- The Institute expects that the use of NRC developed self-curing concrete for the construction or rehabilitation of less than 5% of the current/future deficient bridges or infrastructure in Canada in the coming years should result in a cost-savings of approximately $685 M for the country. These cost savings are expected to results from the reduced maintenance costs as well as the extended service life of the infrastructure.
Although limited evidence was available, some findings indicate that projects such as the energy code, the post-evaluation of green buildings, and the development of energy efficient products should result in cost-savings for Canadian homeowners and building operators. While total savings for homeowners and building operators are difficult to quantify, examples of energy costs saved were identified in the evaluation as part of the CCHT case study. These include:
- A project on the use of domestic hot water heat pump systems demonstrated that these systems can lead to cost savings of $70 to $120 per year in hot water costs for the average household using this system; and,
- A project on advanced integrated mechanical systems showed that although these systems cost more to install than separate systems, their improved performance reduces utility bills by $150 to $200 per year.
Finally, every NRC-IRC research project that contributed to improvements in the durability / sustainability of houses and that has been adopted by the industry or included in the code is likely to have contributed to reduced maintenance costs for homeowners.
NRC-IRC has conducted several multi-partner research projects that have provided science-based evidence in support of the National Fire Code. The evaluation findings show that this research has contributed to the safety of buildings in Canada. As noted by one building regulatory official interviewed as part of the evaluation, "I think the program [i.e., NMC] has contributed [a] huge amount to society. People in Canada and [province name] go to buildings with [the] understanding that they are safe. We are not immune to tragedy in Canada but we have a high level of assurance that buildings are safe."
Research projects on fire safety-related themes, such as fire behavior in multi-suite residential dwellings, the impact of fire on floor/ceiling assemblies in basements, or fire protection standards for accessory apartments in basements, have all influenced the National Fire Code development. NRC-IRC's project on the protection standards for accessory apartments provides a concrete example of how research can improve the safety of buildings. As part of this project, NRC-IRC and its partners examined the need for fire protection measures (e.g., installation of gypsum sheets or sprinklers) to last for a 45-minute period, which was previously required by the code. The research demonstrated that the required 45-minute period was irrelevant given that occupants in such types of fires are overcome with smoke within ten minutes. As a result of the research, smoke detectors in all parts of a house, or between units in multi-family dwellings, was identified as a more appropriate measure than the previous fire protection measure.
With regards to the long term impacts of NRC-IRC on the fire safety of buildings in Canada, data collected by the Council of Canadian Fire Marshals and Fire Commissioners on the number of fires and injuries from fire indicates a steady decline since 1990.Footnote 50 While other factors undoubtedly have a role to play in the number of fires and injuries from fire, the decrease can be attributed, at least in part, to the continuous improvement made to the fire codes over the years in addition to the fact that it is the law in most P/Ts and builders must comply with it.
In addition to the research conducted in support of the NMC (and more specifically, the fire code), NRC-IRC has worked with provincial and municipal governments in Canada and the USA to protect the lives of motorists from fire and smoke resulting from road accidents in tunnels. For example, NRC-IRC simulated a real fire in the Lafontaine tunnel in Montreal to test the ventilation system used to evacuate smoke from the tunnel in case of fire resulting from a traffic accident. The project resulted in the automation of the de-smoking ventilation system as well as increased confidence in the reliability of the system in case of fire.
Quality of Life and Health
While human health is a complex phenomenon that can be explained by multiple exogenous and endogenous factors, some building regulatory officials interviewed as part of the evaluation felt that NRC-IRC has contributed to the health and quality of life of Canadians by ensuring that buildings are safe and of an acceptable quality. Although specific examples could not be identified as part of the evaluation, the overall quality of the built environment in Canada resulting from the code provides an indication that results were achieved in this regard. Research in areas such as mold, volatile organic compounds, ventilation systems and indoor air quality are examples of NRC-IRC projects that are expected to have an impact on the health of Canadians.
Evidence collected as part of the evaluation shows that the acoustic performance of buildings can have a significant impact on the quality of life and health of building occupants (e.g., findings show that noise from adjacent units can cause stress to occupants, which results in health problems). In this regard, NRC-IRC's work on acoustic measurement tools for wood frame buildings is expected to have an impact on the quality of life of building occupants.
The evaluation found evidence to suggest that some of the projects undertaken by NRC-IRC since 2005-06 will, if the right conditions are in place, contribute to positive environmental benefits. More specifically, the adoption of the energy code by the P/Ts should minimize the energy consumption in buildings and, consequently, contribute to a reduction in greenhouse gases. This logic can also be applied to other research projects that NRC-IRC has worked on that have contributed to the reduction of energy consumption by the housing sector in Canada.
It is also expected that NRC-IRC's work on mid-rise wood frame buildings will reduce greenhouse gases by decreasing the amount of concrete used in the built environment. Evidence provided by an external interviewee on the code change made by British Columbia with regards to the height of wood frame buildings indicates that industry has already begun to increase its use of wood in the built environment. Based on this positive uptake, it is reasonable to expect that the use wood across Canada, particularly if there is a change to the NMC, will continue to increase and result in positive environmental benefits.
5. Efficiency and Economy
This section of the evaluation looks at the extent to which the resources allocated to the Institute are being used in an economical manner in producing outputs and progressing towards expected outcomes.
- To what extent has NRC-IRC used its resources in an efficient and economical manner in producing outputs and progressing towards expected outcomes?
- What factors have affected the extent to which NRC-IRC operates in an efficient manner?
- To what extent do code-related complaints affect NRC-IRC's operational efficiency? Does this vary by type of complaint, group or other key conditions?
- NRC-IRC has used its resources in an economical manner between 2005-06 and 2011-12.
- While overall NRC-IRC clients were very satisfied with the services they received, there were some concerns raised with the timeliness of services and deliverables and the cost of services.
- NRC-IRC has increased its productivity in more recent years, and has had in place appropriate management systems, processes and practices to support efficient operations.
- While changes have been made to the delivery of the Institute's activities to enhance efficiency, opportunities still exist to improve the IRC-CCMC product evaluation services and product evaluation and decision-making process.
- The code funding model is an efficient system to ensure the development of the NMC on a five year cycle but provides limited flexibility in terms of funding other types of activities that are not directly linked to the development of the NMC.
Evaluative evidence suggests that NRC-IRC has used its resources in an economical manner between 2005-06 and 2011-12. Over the seven-year period, the financial resources used by the Institute have remained relatively stable (ranging from $28M to $34M).
NRC-IRC was also able to successfully leverage funds in the research projects it conducted. For example, as part of the Special Interest Group for Dynamic Evaluation of Roofing Systems (SIGDERS) project, for every $1 invested by NRC, $5 was contributed by other parties. The high performance concrete project that NRC-IRC engaged in also resulted in the same leverage ratio (5:1). Looking across all collaborative research projects that the Institute engaged in between 2005-06 and 2011-12, NRC-IRC leveraged $3 from other partners for every $1 it contributed.
The Institute's use of joint facilities, such as CCHT, further speaks to their economic use of resources. In fact, CCHT's operational practices are considered a best practice from an economic perspective. At CCHT, the ongoing operating, upgrading and maintenance costs of the facility are funded by revenue from ongoing collaborative R&D projects as well as the fees paid by parties using the facility. The collaborating partners are responsible for equally financing approved annual debt. There are only three CCHT dedicated employees (i.e., two NRC employees working one third of their time on CCHT-related activities and one FTE employed by NRCan); all other resources are brought in as needed for specific projects which limit the ongoing CCHT operational costs.
Finally, 2011 data provided by NRC's Administrative Services and Property Management (ASPM) suggests that, on average, NRC-IRC operated its laboratories in a more economic manner when compared to comparator laboratories. Each year, NRC-IRC spent approximately $4M on the operation of its buildings (i.e., laboratories, service buildings and storage).
In order to assess the extent to which NRC-IRC has been successful at using its resources and converting them into outputs, client satisfaction, productivity, program delivery, management systems, processes and practices, and the code development system were explored.
5.2.1 Client Satisfaction
Client satisfaction with program activities and outputs (i.e., services received, deliverables produced) was used by the evaluation team as an indicator of NRC-IRC's operational efficiency.Footnote 51 Overall, external clients interviewed consulted as part of the evaluation (i.e., companies, government departments, and industry associations) were very satisfied with the services they received from the Institute. As is demonstrated in Table 4, average satisfaction rates on different services areas ranged from very satisfied to satisfied.
|Clients Average Satisfaction Working with NRC-IRC in Terms of…||Average satisfaction|
|…Overall quality of services||Very satisfied|
|…Ability of Construction Portfolio staff to understand your needs||Very satisfied|
|…Scientific / technical competencies of staff||Very satisfied|
|…Quality of data / results||Very satisfied|
|…Confidentiality and security of information||Very satisfied|
|…Availability of staff||Satisfied|
|…Timeliness of services and deliverables||Satisfied|
|…Cost of services||Satisfied|
|…Access to world-class infrastructure and equipment||Satisfied|
Source: Key informant interviews with external clients
Two areas where satisfaction was not as consistently high for all external interviewees were timeliness of services and deliverables as well as the cost of services. Concerns were raised with regards to project delays by some external clients interviewed as part of the evaluation. The project delays that clients spoke of were attributed to insufficient human resources at certain facilities (i.e., the fire test facility), vacation time and loss of experienced staff. In fact, human resource risks were also highlighted by internal interviewees as there have been retirements and sick leaves that the Institute has had to address. Compounding this is the fact that hiring qualified people with specific competencies is often challenging as many usually need to be recruited from outside Canada.
In addition to project delays, external clients' satisfaction with the cost of NRC-IRC services was not consistently as high as other service dimensions. Some interviewees from companies noted that that the cost of NRC-IRC services was high. In some cases, the high cost was thought to be justifiable due to the quality of service. In other cases, the cost was not viewed as a large imposition to the company due to NRC-IRC's ability to identify collaborators for the research project to offset the total costs to one individual company.
Satisfaction with IRC-CCMC was not directly assessed as part of the current evaluation. However, it was explored as part of the 2010 IRC-CCMC Business Review. While a majority of the manufacturers surveyed as part of the Business Review were satisfied with the ease of initiating a request with IRC-CCMC and the responsiveness of IRC-CCMC to the initial request, only 35% were satisfied with the time it takes IRC-CCMC to complete a report. The two elements driving this dissatisfaction were the time it takes to develop a technical guide and the time it takes to receive the final report after successful testing. Concerns were also highlighted as part of the Business Review about the general responsiveness of IRC-CCMC during the product evaluation process. The 2010 IRC-CCMC Business and Governance Review put forth recommendations for IRC-CCMC to improve the evaluation process by clarifying expectations around the product evaluation process (e.g., timeframe, cost, quality) with clients and to improve communications with stakeholders (e.g., client care program) that would address the challenges with client satisfaction. Due to the NRC-wide changes that began in 2009-10, actions towards these recommendations were put on hold by the Institute and have not yet been addressed. As such, opportunities exist for IRC-CCMC to improve its customer experience.
Recommendation 5: Given the importance of the client experience as part of the new NRC strategy and based on the finding that opportunities exist for NRC-IRC to improve the timeliness of its services and deliverables, it is recommended that the Construction Portfolio review ways to improve its service delivery.
The efficiency of the Institute can be assessed by looking at its operational productivity and its research productivity.Footnote 52 On average between 2005-06 and 2011-12, for every NRC-IRC employee, $39K in revenue was generated while for every research and technical staff member, $56K in revenue was generated. As is demonstrated in Figure 11, both measures of productivity decreased somewhat in the earlier years of the evaluation time period, however, have steadily increased in more recent years. This would suggest that the Institute has increased its efficiency over the recent years, primarily because of an important increase in revenue that can be attributed to the new strategic direction of the organization. It has to be noted that the ability to attract external investment may have been affected by the world economic crisis that lasted from 2008 to 2010.
Figure 11. NRC-IRC Productivity between 2005-06 and 2011-12 Source: NRC-IRC Administrative Data
Long description of NRC-IRC Productivity between 2005-06 and 2011-12 Source: NRC-IRC Administrative Data
|Operational productivity ratio||$44,000||$41,001||$37,000||$33,001||$32,000||$37,000||$49,000|
|Research productivity ratio||$62,000||$58,001||$51,000||$48,001||$46,000||$53,001||$70,001|
5.2.3 Program Delivery
Over the course of the evaluation time period, NRC-IRC has made various operational changes in an effort to increase their efficiency. Over the past three years, in accordance with the establishment of NRC's new strategic direction, NRC-IRC engaged in a re-alignment exercise that involved assessing their competencies, determining gaps, and identifying opportunities to streamline activities. As a result of this exercise, certain areas of activities were abandoned in early 2013 in an effort to increase efficiency (i.e., CSIR in Regina, Saskatchewan and the Computer-Assisted Construction Technologies Program in London, Ontario).
Institute records indicate that NRC-IRC has taken steps to reduce and eliminate the backlog at IRC-CCMC, providing further evidence of operational changes for enhanced efficiency. However, opportunities to improve and streamline the delivery of IRC-CCMC services that were originally identified in the 2010 IRC-CCMC Business and Governance Review, were also echoed by internal interviewees as part of the current evaluation. The 2010 IRC-CCMC Business and Governance Review specifically put forward a series of recommendations around reviewing the product evaluation services and improving the product evaluation and decision-making process. This included developing an approach to identify the level of risk associated with an innovative product to calibrate the evaluation processes accordingly. As such, opportunities to improve the efficiency of IRC-CCMC product evaluation services are still present.
Recommendation 6: In order to optimize the efficiency of IRC-CCMC delivery, it is recommended that the Construction Portfolio implement the recommendations put forward in the 2010 IRC-CCMC Business and Governance Review.
5.2.4 Management Systems, Processes and Practises
The presence of sound management systems, processes and practices are factors that have positively affected the efficiency of NRC-IRC's operations. NRC-IRC has had a Project Management System in place, including a Personnel Cost Planning System, for over 10 years. While there was some indication from internal key informant interviewees that the project management system may not have been used consistently by all programs or projects in the Institute, going forward this will change with the establishment of NRC's new strategic directions, organizational requirements and the formation of the NRC Program and Project Support Branch .
Additionally, NRC-IRC has maintained a Client Information System (CIS) to manage its financial transactions and relationship with clients. This system contained valid and reliable information on clients and projects. In support of the code development process, NRC-IRC used a Content Management System that keeps track of such things as changes to the code provisions and comments from the public consultation process. According to one internal interviewee, this system resulted in efficiencies to the code development process.
Finally, contributing to NRC-IRC's efficiency is its risk management practices. A 2010 audit of NRC's risk management practices found that NRC-IRC has acceptable risk management process, methods and tools; risk identification practices; and risk assessment. Some areas for improvement were identified, including the Institute's control assessment and its risk response and action plan.
5.2.5 Efficiency of the Code Development System
In addition to the overall efficiency of NRC-IRC, the evaluation explored the efficiency of the code development system. This section begins by providing an overview of the funding model for the code development system, followed by a discussion on the efficiency of the funding model and the impact of code-related complaints on NRC-IRC's efficiency.
Funding of the Code Development System
NRC is responsible for the administration of the revenues/costs related to the code development system at the national level. As such, at the beginning of each five-year code cycle, NRC-IRC determines the estimated costs required to fund the activities associated with the development of the NMC that were agreed-upon by the partners. These activities include but are not limited to travel expenses for the CCBFC and committee meetings, administrative costs associated with the operation of the IRC-CCC (e.g., CCBFC membership database), the cost of translating and printing the codes, as well as costs related to code sales and marketing.
According to NRC-IRC, the cost of developing the 2010 version of the NMC was estimated at approximately $42M. This includes (based on a five-year cycle):
- Code sales to clients (approximately $30M); and,
- NRC-IRC A-base funding (approximately $12M).
Overall, the main contributor to the codes is the private sector; they not only are the primary purchaser of codes, but they also provide a notable in-kind contribution through their roles on the CCBFC. Footnote 53 For their part, the P/Ts do not provide a direct up-front contribution to the code development process; however, they grant NRC-IRC with the rights, through various financial arrangements, to collect revenues from code sales.Footnote 54 While a majority of the P/Ts do participate in the national code development system, there is an exception – Ontario. Despite its participation in the CCBFC, Ontario maintains its own code development system in parallel with the national system; NRC therefore does not generate revenues from Ontario code sales. This was identified by several internal and external interviewees as a "major irritant" and a financial constraint to the cost-sharing model for the code development. Ontario does, however, contribute to the code development system in other ways, such as sharing training material on codes and through its direct contribution to R&D projects in support of code development. That said, these were deemed insufficient by all internal and external interviewees consulted.
Efficiency of the Code Funding Model
Almost all building regulatory officials from the P/Ts interviewed expressed a strong level of satisfaction with the current code funding model. Interestingly, almost half of these interviewees were either not aware of the current funding model used to support the development of codes or had difficulty answering questions related to the efficiency of the cost-sharing model. While this finding was unexpected considering the role of these individuals (i.e., senior building regulatory officials), it may be explained by the fact that they have limited involvement in the administration of the funds, and that most P/Ts do not contribute financial resources directly to the code development system (as was explained earlier).
Another key finding is that the resources available from the current code funding model support NRC-IRC exclusively in the development of the model codes and nothing else. This was found to be challenging for NRC-IRC and the CCBFC, particularly considering that a few external interviewees felt that the responsiveness of the national code system could be improved to better meet their needs. In order to increase responsiveness, more resources would certainly be needed. As an example, the P/Ts had identified a need for an Energy Code, but did not have the resources to develop it. Likewise, NRC did not have the resources. As a result, NRCan stepped up to fund its development. The limited resources to support the code development system also explains why NRC-IRC has not undertaken activities such as the development of building code training programs/tools/materials for building officials. As was discussed previously, the training of building officials has a significant impact on the use of innovative products / materials in the built environment.
In sum, the code funding model is an efficient system to ensure the development of the NMC on a five year cycle but provides limited flexibility in terms of funding other types of activities that are not directly linked to the development of the NMC.
Impact of Code-related Complaints on NRC-IRC
The impact of code-related complaints on the operational efficiency of NRC-IRC was briefly explored. Overall, discussions with internal interviewees did not point to any major issues in this regard. In fact, these interviewees noted that there is a formal process at the CCBFC to address the concerns of industry or Canadian citizens as part of the code development process. With many stakeholders having a strong interest in the code content, some have used their political influence to circumvent the formal feedback process when the code does not reflect their needs. This type of intervention usually results in code-related complaints directed at NRC-IRC as opposed to the CCBFC. Sometimes, this is due to the incorrect perception that the Institute, as opposed to the CCBFC, decides the content of the code. Given this, NRC-IRC will have no choice but to continue to deal with code-related complaints.
Overall, the Institute has demonstrated a strong value for money for Canada as shown by its clear relevance and its ability to generate impacts in an efficient and economical way (i.e., performance). The evaluation found that NRC-IRC was able to address the current and future needs of its clients in specific niche markets, that it is consistent with the roles and responsibilities of the federal government, and that it is strongly aligned with the government and NRC's priorities. With regards to the performance of NRC-IRC, the evaluation found that the Institute has produced many outputs that were strongly valued by its clients and stakeholders. In terms of impacts, NRC-IRC has contributed to reduced barriers to market access and to the growth of some firms, cost-savings for its clients and stakeholders, improved fire safety, enhanced health and quality of life for Canadians and is expected to yield environmental benefits in the future. However, instances of the longer-term impacts (i.e., the health and quality of life of Canadians and environmental benefits) were more difficult to capture as part of the evaluation, due in part to the complexity of the construction sector and the diversity of activities undertaken by NRC-IRC. Finally, the evaluation found that the Institute used its resources in an economic and efficient manner. The presence of appropriate management systems, processes and practices were found to play a key role in the Institute's efficient operations.
In 2010-11, the Institute began its transformation to a Portfolio (the NRC Construction Portfolio)Footnote 55, whereby programs and operations were re-oriented and focused on industry and government needs and outcomes. While the evaluation did not focus on the effect of this transition on NRC-IRC's relevance and performance, findings from some lines of evidence (e.g., document review and key informant interviews) did touch briefly on the transition. Overall, information on the new NRC Construction Portfolio suggests that it is well positioned to increase its value to Canada. For example, efforts have been made to address some of the challenges faced by IRC-CCMC (e.g., project backlog, pricing policy, integrated role in technology deployment and commercialization path in every programs) and the Portfolio's activities have been refocused to a small number of key research areas that address the current needs of industry (e.g., energy efficiency of commercial buildings, height of wood-frame buildings). In light of the new NRC mandate to focus on the growth of the Canadian industry, the NRC Construction Portfolio's ability to engage key private and public sector partners on an on-going basis will be critical to the future success of the Portfolio.
Despite the evaluation's focus on the Institute as opposed to the Portfolio, the evaluation was able to identify opportunities for improvement that are expected to yield beneficial outcomes for the new NRC Construction Portfolio. More specifically, the evaluation identified opportunities for the new Portfolio, to: better communication its new strategic orientation to OGDs, better define its targeted clients; improve the timeliness with which it provides services; improve the services of IRC-CCMC by implementing the recommendations from the IRC-CCMC Business and Governance Review; and, assess the alignment of CCHT with the Portfolio's new strategic objectives to invest resources appropriately. Finally, in the area of the National Model Codes, opportunities were identified for the new Portfolio to work with the provinces and territories to reduce code adoption delays and to garner the financial support of Ontario in order to increase the efficiency of the national code development system.
7. Management Response
|Recommendation||Response and Planned Action(s)||Proposed Person(s) Responsibilities||Timelines||Measure(s) of Achievement|
|Recommendation 1: NRC should take the appropriate means to ensure that other government departments are aware and understand the new strategic direction of the organization and how the new NRC Programs and services can support their mandate.||Accepted
Since April 1, 2013, Construction management has been implementing plans to communicate the new strategic directions to stakeholders, including other government departments, and how NRC programs, competencies, and facilities can support them in their mandate.
Supported by Portfolio Business Advisor
Agreements signed with OGDs for projects and technical services.
|March 2014||Communication plan developed and executed to support synergy of mandate between relevant OGDs and NRC Construction.|
|Recommendation 2: The new Construction Portfolio, should seek ways to increase its understanding of its potential targeted clients (i.e., innovative firms in the construction sector), including those outside of Ontario and Quebec.||Accepted
The management recognizes the need to further implement new strategies to understand the needs of its targeted and evolving clients. The new NRC Programs brought the opportunity to implement some measures, in alignment with such a recommendation. The management will ensure that it will sustain these measures, including client visits and market analysis, across Canada
Supported by Portfolio Business Advisor
|Business outreach strategy that engages clients from across Canada and linked to current and future plan development.|
|March 2014 (existing programs)
March 2015 (future programs)
|Innovation needs and capacity of targeted clients from existing and future potential programs across Canada identified through market analysis and client visits.|
|March 2016||Contracts more balanced with clients and partners across Canada.|
|Recommendation 3: Given that CCHT supports the needs of companies to assess the performance of their products and technologies, that it has contributed to NRC-IRC's ability to increase its outreach to the private sector and that further investment will be required to update the facility, it is recommended that the Construction Portfolio assesses the extent to which the facility is aligned with the strategic objectives of NRC , and if it is deemed a priority, work with CCHT stakeholders to invest the appropriate resources to ensure CCHT's continued relevance to industry.||Accepted
The management agrees with the need for a CCHT assessment respective to its alignment with the NRC programs and strategic direction. The outcome of this assessment and the anticipated program demands on CCHT will determine the way forward toward its future mode of operations as well as mid to long-term investments by NRC and the other CCHT stakeholder organisations.
Supported by Intelligent Building Operations Director
||Agreement with CCHT partners to develop CCHT assessment and business case for investment.|
|Sept 2014||If business case can be justified then submit investment for approval by SEC.|
|March 2014||Increased CCHT use by NRC clients.|
|Recommendation 4: In order to accelerate the adoption of the National Model Codes by all Provinces and Territories, it is recommended that the new Construction Portfolio:
a) seek ways to increase the engagement of Provinces and Territories; and,
b) take a leadership role in consulting with the CCBFC and PTPACC to coordinate/streamline the public consultation process.
|Accepted in principal
The management agrees with the finding that a faster and comprehensive adoption of building codes would be a desirable outcome. It will therefore undertake efforts, and engage Commission, Provinces, and Territories to find ways and take leadership as recommended while adhering to all constitutional aspects of Federal, Provincial, and Territorial responsibilities.
Supported by Building Regulations Director
||Codes sales improvement action plan developed and implemented.
|March 2015||PTPACC and CCBFC agreement to develop plan for more engagement of P/T and coordinated approach of public consultation process.|
|Dec 2015||2015 Construction Codes adoption action plan implemented.|
|Recommendation 5: Given the importance of the client experience as part of the new NRC strategy and based on the finding that opportunities exist for NRC-IRC to improve the timeliness of its services and deliverables; it is recommended that the Construction Portfolio reviews ways to improve its service delivery.||Accepted
The management is confident that NRC's new business model based on diligent program and project management will lead to operational efficiencies and a timely provision of services and deliverables to clients. Program and project delivery are periodically monitored; as well as customer satisfaction surveys and follow up action.
Supported by Operations Director
||KPI's on project management and client satisfaction regularly monitored and met.|
|March 2014||Specific service delivery monitoring developed as part of the program/project review process.|
|Recommendation 6: In order to optimize the efficiency of IRC-CCMC delivery, it is recommended that the Construction Portfolio implement the recommendations put forward in the 2010 IRC-CCMC Business and Governance Review.||Accepted
The management will undertake efforts to accelerate the implementation of the recommendations made in the 2010 review. Changes to NRC's new business model will be taken into account accordingly.
Supported by Building Envelope and Materials Director
||CCMC manager hired (based on recommended competency profile).|
|Sept 2015||Business/governance recommendations implemented, including business model, training and recruiting.|
Appendix A: Evaluation Matrix
|Core Issues||Evaluation Questions||Review of Administrative / Financial data||Document and Literature Review||Key Informant Interviews||Case Studies|
|Continued Need for Program||R1. What are the needs of NRC-IRC stakeholders?||X||X|
|R2. To what extent are the activities of NRC-IRC aligned to current and future needs of the Construction Portfolio's stakeholders?||X||X|
|Alignment with Federal Roles and Responsibilities||R3. To what extent are the activities of NRC-IRC consistent with federal roles and responsibilities?||X||X||X|
|R4. Is there a justifiable role for the federal government and, specifically, NRC in supporting the development of construction code?||X||X||X|
|Alignment with Government and NRC Priorities||R5. To what extent are NRC Construction's activities aligned with federal government priorities and NRC's strategic outcomes?||X||X|
|Achievement of Expected Outcomes||P1. To what extent and in what conditions has NRC-IRC been successful at reaching its targeted clients and collaborators?||X||X||X||X|
|P2. To what extent and in what conditions has NRC-IRC contributed to the harmonization of Construction Codes and accepted practices across Canada?||X||X||X|
|P3. To what extent and in what conditions has NRC-IRC contributed to the commercialization and/or increased market access for Canadian products and technologies?||X||X||X||X|
|P4. To what extent and under what conditions has the use/adoption of objective-based construction codes increased the use of innovative design or the adoption of innovative material and technology in the Canadian built environment?||X||X||X|
|P5. To what extent and in what conditions have NRC-IRC's activities contributed to the health, safety and accessibility to building for Canadians?||X||X||X|
|Demonstration of Efficiency and Economy||P6. To what extent has NRC-IRC used its resources in an efficient and economical manner in producing outputs and progressing towards expected outcomes?||X||X||X||X|
|P7. What factors have affected the extent to which NRC-IRC operates in an efficient manner?||X||X||X||X|
|P8. To what extent do code-related complaints affect NRC-IRC's operational efficiency? Does this vary by type of complaint, group or other key conditions?||X||X|
Appendix B: Methodology
This section presents the detailed evaluation methodology, and includes a discussion on the evaluation design and approach, the scope, the challenges and limitations, and the methods.
The evaluation was carried out in accordance with NRC's approved evaluation plan and Treasury Board Secretariat (TBS) policies. The last evaluation of NRC-IRC took place in 1998. The selection of methods was based upon the most efficient means of addressing the evaluation issues in a rigorous way, while taking into account cost, time and resource constraints, as well as other considerations, such as evaluation scope, evaluation budget and minimizing response burden. The evaluation approach and level of effort was commensurate with the program risk, which was assessed as medium during an assessment conducted as part of the planning phase.
In order to maximize the possibility of generating useful, valid and relevant evaluation findings, mixed methods were used for this evaluation, allowing for triangulation (i.e., convergence of results across lines of evidence) and complementarity (i.e., developing better understanding by exploring different facets of a complex issue). The specific methods used in the study are described below.
The evaluation scope corresponded to the period between the publication of the first objective-based National Model Code (NMC; i.e., 2005) and the transformation of the Institute into a Portfolio (i.e., April 2012). The transition from a prescriptive-based to a type of objective-based model code represents a significant change to the code development system and for this reason the evaluation timeframe was aligned to the date that corresponded to this transition.
Challenges and Limitations
Various limitations and challenges were experienced in conducting the evaluation of NRC-IRC that need to be taken into consideration when interpreting the findings. The challenges and limitations in the current evaluation are not unique; rather, they are common in most evaluations conducted within the federal government. In order to alleviate the effects of the limitations and challenges on the evaluation findings, various mitigation strategies have been used. Table 5 presents the limitations and challenges, their potential effect on the project, as well as the mitigation strategies used to minimize their impact.
|Limitations and Challenges||Impact on Evaluation Project/Findings||Mitigation Strategies|
|Interviews as primary line of evidence.||Interview data are based on personal perceptions of evaluation participants, and may appear to be subjective.||
The interview results were triangulated against the findings from other lines of evidence (e.g., literature review, review of administrative databases).
In the case where statements from interviewees could be verified, evidence was requested, particularly in the case of claims of impacts.
|Limited representation of private sector companies in external interviews.||There is a potential bias towards the perspectives of government department departments, internal stakeholders and building code regulators.||One of the key strategies that was use to collect the views of the private sector was to interview key industry associations, which represent the private sector, in targeted sectors.|
|Coverage of all NRC-IRC activities, services and projects.||Risk of not capturing all the impacts of the NRC-IRC.||NRC-IRC is one of the largest institutes at NRC, with a large diversity of activities and services. Given the limited evaluation resources available, a risk-based planning approach was used to identify key priorities for the evaluation to meet the needs of both NRC and NRC-IRC management, and the 2009 Treasury Board Secretariat (TBS) Policy on Evaluation. (i.e., the five core evaluation issues).|
|Providing an evaluation judgment on the performance of the Institute as it pertains to reach||From an evaluation perspective, two options can be used to determine the performance of a program in terms of its ability to reach its targeted clientele. The first is to compare the number of clients reached to the targeted number of clients. In the case of NRC-IRC, targets were not determined by the Institute for the period covered by the evaluation. The second option is to compare the number of clients reached to the number of potential clients, in this case, the construction sector. However, challenges arise in defining the construction sector, and not all companies within the sector are potential clients.||Given this, the evaluation focused on describing the clients reached.|
|Attribution of longer term impacts of NRC-IRC, especially in the case of R&D projects addressing sector specific needs rather than individual firms needs.||NRC has engaged in many long term projects that involved several stakeholders with expected outcomes targeted at large segments of the construction industry. This made it challenging to consult all stakeholders affected by a NRC-IRC project.
The evaluation's ability to measure impacts was also influenced by the fact that most of the outputs produced by NRC-IRC take a long time to yield outcomes. These long term outcomes are also greatly diffused within the construction sector, making their measurement challenging.
Case studies were conducted on large scale NRC-IRC projects to illustrate the impacts of the Institute. As part of the case studies, a large and diverse number of stakeholders were consulted to capture the range of impacts from these projects.
To ensure that longer-term impacts could be measured, projects initiated prior to the period covered by the evaluation were selected.
A theory-base evaluation approach was also used to contextualize the evaluation findings by assessing the factors and conditions that explain the achievement of impacts.
|As a result of NRC's new strategic and organizational direction (i.e., transformation of NRC-IRC to the Construction Portfolio), the evaluation object was in a state of change at the time of the evaluation.||The strategic orientation, organizational structure and operations of NRC-IRC were affected by NRC's new strategy in 2011-12. While the evaluation looked retrospectively at the performance of the Institute, recommendations needed to be framed in light of the new organizational structure and operations. It was challenging to ensure an appropriate blend of retrospective findings and prospective recommendations. This may potentially limit the relevance and, consequently the use, of the evaluation products by senior management.||In order to ensure that the evaluation recommendations were relevant, members of the Evaluation Advisory Committee, the Construction Portfolio's General Manager and the NRC VP Office, Engineering, were consulted.|
|Reliability and validity of the NRC-IRC client databases.||Erroneous or missing data in administrative databases can affect the reliability and validity of results from that line of evidence.||The evaluation met with the NRC-IRC contracting office to assess the quality of their client databases. The data were deemed valid and reliable, primarily because one specific individual was dedicated to the administration of the database during the time period covered by the evaluation. The data were also assessed by the evaluation team prior to conducting the analyses, and were found to be valid and reliable (e.g., missing data were minimal; there were limited out of range values).|
|Risk of errors resulting from data manipulation to the client database or misinterpretation of information contained in the database.||The two client databases used on the evaluation - the Client Information System (CIS) and the IRC-CCMC client database - were modified for use in the evaluation. In doing so, there is a risk that manipulation errors may have occurred. Likewise, interpretational errors as to what the data represented may have occurred. Both potential risks could affect the conclusions drawn from this line of evidence.||
All data manipulations and analytical decisions were documented to ensure that the analysis could be replicated.
NRC-IRC staff responsible for the databases were contacted periodically to clarify any questions or issues related to the interpretation of the information in the databases.
Findings from the analysis of the client databases were validated by the Institute as part of the factual review of the final evaluation report.
The evaluation used both qualitative and quantitative research methods. The specific methods used in the study included:
- Literature and document review;
- Administrative and performance data review;
- Key informant interviews; and
- Case studies.
Literature and Document Review
Internal and external documents were reviewed, synthesized and integrated into the evaluation to provide context and history, and contributed to the analysis of relevance and performance. The review of internal documents included strategic and business plans, performance reports, presentations, previous evaluation studies, other internal studies and reports and audit reports. The review of external documents included those produced by other government departments and central agencies, and published journal articles.
A literature review on performance-based and objective-based approaches to building regulation was conducted as part of the planning phase of the evaluation project. The literature review was used to provide context and to answer evaluation questions related to what makes performance-based approaches to regulations work, for whom, under what conditions and why.
Analysis of Administrative and Performance Data
Financial and human resource data were reviewed to provide context to the findings obtained through other lines of evidence. Administrative and performance data were also analyzed and answered evaluation questions related to client reach, outcomes and efficiency of space-use. Key administrative and performance data sources used included:
- Data on clients from the Client Information System (CIS) database;
- Data on clients from the IRC-CCMC client database;
- Information about code sales; and
- Data on building operational costs, space-use, and building condition from NRC-Administrative Services and Property Management (ASPM).
Key Informant Interviews
Internal and external stakeholders were consulted to collect qualitative evidence with regards to the relevance and performance of NRC-IRC. The information gathered through the qualitative, semi-structured interview process was based on personal experiences, opinions and expert knowledge. This information played an important role in contextualizing performance data and other statistics. In all, 41 key informant interviews were conducted with NRC management and staff, clients and collaborators and other external stakeholders involved in the construction sector (see Error! Reference source not found.).
Clients and collaborators
Government departments (federal, provincial, municipal)
External stakeholders involved in the construction sector
Representatives from Provincial / Territorial Building Regulatory Agencies Footnote 56
A list of potential interviewees was produced with the assistance of NRC-IRC. Interview guides were developed for each respondent group and/or interviewee. Following an initial email sent by NRC-IRC, the evaluation team followed-up with key informants to secure their participation and schedule an interview. Interviewees were sent a copy of the interview guide in advance of the interview. Interviews were conducted by phone. Interviews were conducted in the preferred official language of the interviewee and lasted between 30 and 90 minutes, depending on the interviewee group.
A series of five case studies were conducted as part of the evaluation to gain a better understanding of the impacts resulting from NRC-IRC projects as well as to identify the critical factors and conditions that facilitate or impede the achievement of impacts and to gain a better understanding of the role played by NRC-IRC in the Canadian construction innovation system. The case studies were chosen in collaboration with NRC-IRC management and sought to capture an appropriate representation of the diverse impacts NRC-IRC has had on the construction industry and public good. As part of each case study, key documentation was reviewed and interviews were held with relevant internal and external stakeholders. Upon completion of a case study, the report was shared with the NRC-IRC project lead responsible for the project for validation. The five case studies included (n = interviews conducted per case study):
- National Energy Code (n = 7)
- North Channel Bridge (n = 5)
- Special Interest Group for Dynamic Evaluation of Roofing Systems (SIGDERS) (n = 6)
- Sound Insulation Wood frame (n = 6)
- Canadian Center for Housing Technology (CCHT) (n = 15)
In addition to the five case studies conducted as part of the current evaluation, two case studies that were conducted as part of a Natural Resources Canada (NRCan) led evaluation were reviewed and considered as part of the current evaluation. While these case studies covered projects that were funded by NRCan, all of the R&D work was performed by NRC-IRC. These case studies were on the following NRCan research projects: post-evaluation of green building and solid state lighting.
Appendix C: Adoption of the 2010 National Model Code
|Province/Territory||National Building Code 2010||National Fire Code 2010||National Plumbing Code 2010|
|Yukon||Adopted April 2011||Adopted April 2011||Adopted April 2011|
|Northwest Territories||Adopted April 2011||Adopted April 2011|
|Nunavut||Expected when Building Act is adopted in 2014||Expected when Building Act is adopted in 2014|
|British Columbia||Adopted Fall 2012||Adopted Fall 2012||Adopted Fall 2012|
|Alberta||Expected in 2014||Expected 2014||Adopted September 2012|
|Saskatchewan||Adopted May 2013||Expected in 2013||2005 version|
|Manitoba||Adopted April 2011||Adopted December 2011||Adopted April 2011|
|Quebec||In progress||Adopted Fall 2012||In progress|
|New Brunswick||Adopted January 2013||Adopted January 2012||Adopted January 2013|
|Nova Scotia||Adopted June 2011||Adopted June 2011||Adopted June 2011|
|Prince Edward Island||Adopted 2011: Summerside, Charlottetown, Stratford||TBD||2005 version|
|Newfoundland and Labrador||Adopted except NBC Part 9 for 1 and 2 dwelling units||Adopted|
Sources: Internal NRC Construction Portfolio Documents (Updated October 2012) and http://www.nationalcodes.nrc.gc.ca/eng/code_adoption.html
Appendix D: NRC-IRC Outputs
This table provides an overview of the main categories of outputs produced by NRC-IRC in addition to specific examples.Footnote 57
|Type of Output||Examples|
|Knowledge creation||Knowledge gained in areas related to:
|Test results from compliance / performance assessments of products and materials||
|New test methods and protocols||
|Industry guidelines and practical tools for industry||
|Development of new/improved products and technologies||
|Tools in support of decision-making||
Appendix E: Selection of Documents Reviewed
Bergeron, D., (n.y.). Codes for Existing Buildings: Different Approaches for Different Countries. National Research Council.
Bergeron, D., Bowen, B., Tubbs, B., and Rackliffe, T. (2001). Acceptable Solutions. CIB World Building Congress, Wellington, New Zealand, (#257).
Bergeron, D.; Desserud, R.J.; Haysom, J.C. (2004). The Origin and Development of Canada's Objective-Based Codes Concept.
Bukowski, R. (2003). The Role of Standards In a Performance-based Building Regulatory System. Proceedings of the CIB-CTBUH International Conference on Tall Buildings, 8-10 May 2003, Malaysia.
Brio Conseils. (2010). Canadian Construction Materials Centre (IRC-CCMC) Business Review, Final Report and Recommendations.
Canadian Construction Association. (2012). Submission to the Federal R&D Review Panel
Canadian Home Builders Association (n.y.). What will Objective-Based Codes mean for me? Information about Information about Canada's proposed Objective-Based Building Codes for new home builders, renovators and other professionals in the residential construction industry.
Canadian Institute of Plumbing and Heating. (2012). Value Proposition from the CIPH to facilitate the harmonization of market entry required standards and certification schemes
Coglianese, C., Nash, J., Olmstead, T. (2002). Performance-Based Regulation: Prospects and Limitations in Health, Safety and Environmental Protection. Regulatory Policy Program, Center for Business and Government. John F. Kennedy School of Government, Harvard University Cambridge, Massachusetts.
Di Lenardo, B.; Cécire, L.; Poirier, G.; Waters, R. (2005). The Benefit of Intended-Use (Performance-Based) Standards In Support Of Objective-Based Codes.
Deroukakis, E., (2000). Performance-Based Code: Impact on International Trade. Occasional Paper Inter-Jurisdictional Regulatory Collaboration Committee.
Edgar, E. (2010). Governance Review of the Canadian Construction Materials Centre of the National Research Council. Institute of Governance (phase 3 and 2).
Environment Canada. (2012). Canada's Emission Trends 2012
Federation of Canadian Municipalities. (2007). Danger Ahead: The coming collapse of Canada's municipal infrastructure.
Fraser Institute (Jones, L. and Graf, S.). (2008). Canada's Regulatory Burden – How many regulations at what cost?
Government of Canada. (2007), Mobilizing Science and Technology to Canada's Advantage (also known as the S&T Strategy)
Government of Canada. (2011). The next phase of Canada's economic action plan: A low-tax plan for jobs and growth (Budget 2011)
Government of Canada. (2012).Jobs, growth and long-term prosperity: Economic action plan 2012 (Budget 2012)
Government of Canada. (2013). Jobs, growth and long-term prosperity: Economic action plan 2013 (Budget 2013)
Hardie, M. and Newell, G. (2011).Factors influencing technical innovation in construction SMEs: an Australian perspective. Engineering, Construction and Architectural Management, Vol. 18 (6), pp. 618 – 636.
Holmen Enterprises Inc. (2002). Innovation in the Housing Industry – Literature Review in support of examination of innovation in the housing sector: Phase 2 report
Institute on Governance. (2008). A Review of Governance Arrangements of the Canadian Commission on Building and Fire Codes.
Inter-jurisdictional Regulatory Collaboration Committee –IRCC. (2003). Global Policy Summit on the Role of Performance-Based Building Regulations in Addressing Societal Expectations. International Policy and Local Needs Summit Report. National Academy of Sciences Washington, DC USA.
Inter-jurisdictional Regulatory Collaboration Committee - IRCC. (2010). Regulatory Systems Principles and Experiences. A Report of the Inter-Jurisdictional Regulatory Collaboration Committee. Editor Meacham, B.
May, P. J. (2003). Performance-Based Regulation and Regulatory Regimes. Center for American Politics and Public Policy. University of Washington. Paper prepared for the Global Policy Summit on the Role of Performance-Based Building Regulations in Addressing Societal Expectations, International Policy, and Local Needs. National Academy of Sciences, Washington DC, November 3-5, 2003.
May, P. (2003). At the Regulatory Front Lines: Inspectors' Enforcement Styles and Regulatory Compliance. Journal of Public Administration Research and Theory, Vol. 13, no. 2, 117–139.
Meacham, B. (2010). Accommodating innovation in building regulation: lessons and challenges. Building Research & Information, Vol. 38(6), 686–698
Meacham, B., Tubbs, B., Bergeron, D., Szigeti, F., (2002). Performance System Model – A Framework for Describing the Totality of Building Performance.
Moyes, R. (2005). Canadian Home Builders Association Discussion Paper on the Canadian Construction Materials Centre and Code Conformity Assessment.
Mumford, P. (2010). Enhancing Performance-Based Regulation: Lessons from New Zealand's Building Control System.
National Research Council, Institute for Research in Construction. (2008). Business Plan 2009/10-2011/12
National Research Council, Institute for Research in Construction. (2009). Objective and Performance-Based Codes: Impacts and Lessons Learned
Potworowski, A. (2010). Making it Happen – The Transition to a Sustainable Society
Case Study: The Transformation of the National Building Code of Canada: from Prescriptions to Objectives. Telfer School of Management.
Statistics Canada. (2006). The Wealth of Canadians: An Overview of the Results of the Survey of Financial Security 2005, Pension and Wealth Research Paper Series.
Statistic Canada. Gross Domestic Product by Industry 2002-2011. Retrieved on October 1, 2012 from Canadian Industry Statistics - GDP.
Surprenant, D. (2010). The National Building Code of Canada: A Tool for Recovery in the Forest Industry. Library of Parliament. Publication No. 2010-27-E.
Treasury Board Secretariat. Departmental Performance Plans. 2010-2011. Retrieved on October, 1, 2012, from http://www.tbs-sct.gc.ca).
vonWeller, A., (n.y.). Why Building Regulations?
Winter, S., May, P. (2001). Motivation for Compliance with Environmental Regulations. Journal of Policy Analysis and Management, Vol. (20), (4), 675-698.
Zuccollo, J. and Hensen, M. (2012). Weathertight buildings. What lessons can be drawn? NZIER Authoritative Analysis Conference Paper. Palmerston North. Retrieved on October 27, 2012 from