Audit of the industrial research assistance program in the interim operating environment - June 2015

Executive Summary and Conclusion

Background

This audit report presents the findings of the National Research Council Canada's (NRC) Audit of the Industrial Research Assistance Program in the Interim Operating Environment.

Audit Objective

The objective of the audit was to provide just-in-time independent assurance to NRC Senior Management that controls remained effective in the Interim Operating Environment (IOE) to allow NRC to make necessary corrections before fiscal year-end.

Raison d'être

In July 2014, a cyber-intrusion led to the shutdown of NRC's IT network and systems. NRC then implemented Interim Operating Environment (IOE) controls to enable the organization to continue to deliver services and value to clients and the Canadian public. Four audits of the IOE were approved by the President outside of the NRC 2014-15 to 2016‑2017 Risk-Based Internal Audit Plan. These audits under the IOE are: Expenditure Management, Industrial Research Assistance Program (IRAP), Acquisition Cards and Procurement and Contracting.

Audit findings are presented within the context of a compromised operating environment with interim business continuity measures until which time a new network and steady-state business processes are in place.

The Industrial Research Assistance Program is a key pillar of the Government of Canada's strategy to support small and medium sized businesses in the commercialization of research and development efforts. The Program represents nearly a third of NRC's FY2015 operating budget and its continuity within the post-cyber-intrusion environment was imperative to demonstrate financial probity and stewardship of transfer payments for the benefit of Canadians.

Audit Opinion and Conclusion

Overall we found that interim environment controls were sufficient to demonstrate due diligence in the awarding of contribution funding and in the management of contribution claims. The interim, paper dependent, system is not sustainable over the long-term but is sufficient to maintain business continuity until NRC can implement a secure IT environment. We noted that efforts are ongoing as of April 2015 to return to steady-state business processes which are expected to rectify some of the issues identified through the audit.

Summary of Recommendations

No recommendations were identified in the course of the audit. As the control environment will change as NRC moves towards implementing a new, secure network, we expect the use of interim controls to be a temporary action to ensure the continuity of business activities until which time steady-state business processes are introduced.

Statement of Conformance

In my professional judgment as the Chief Audit Executive, sufficient and appropriate audit procedures have been conducted and evidence gathered to support the accuracy of the audit opinion and conclusion. The audit conforms to the Internal Auditing Standards for the Government of Canada, as supported by the results of the quality assurance and improvement program.

Alexandra Dagger, CIA, Chief Audit and Evaluation Executive

NRC Audit Team Members:

Jean Paradis, CPA, CA, CIA, Audit Manager
Andy Lang, CPA, CMA, CIA Senior Auditor

1.0 Introduction

The 2014-15 Audit of the Industrial Research Assistance Program under the IOE was approved by the President outside of the NRC 2014‑2015 to 2016‑2017 Risk-Based Internal Audit Plan following a cyber-intrusion that resulted in the shutdown of NRC's IT network and systems.

1.1 Context

The Industrial Research Assistance Program (IRAP) stimulates wealth creation in Canada through the support of technological innovation. The Program provides support, through advisory and financial expertise, to assist small and medium-sized enterprises at all stages of the innovation process to build their innovation capacity.

Within the Program's portfolio are contributions to firms and organizations, contributions under the Youth Employment Program (YEP) on behalf of Employment and Social Development Canada (ESDC), the Business Innovation Access Program (BIAP), the Canadian Accelerator and Incubator Program (CAIP), and the Canadian HIV Technology Development (CHTD) Program. Despite the cyber-intrusion, NRC‑IRAP was able to disburse 93.2% of its approved authorities in FY2014‑15 which is in line with historical trends.

Following the July 2014 cyber-intrusion, management risk tolerances were adapted to an environment of curtailed authorities and system access rights to maintain the integrity of NRC's network and security. The cyber-intrusion resulted in the shutdown of NRC's financial management system and IRAP's contribution management system, SONAR. NRC and Shared Services Canada (SSC) continue to work towards bringing full system capabilities back online. In October 2014, SONAR was reactivated but with limited capabilities. In the interim, IRAP returned to a paper-based management system relying on legacy workflow processes that were retired when SONAR was upgraded in FY2013. As of April 2015, efforts are ongoing to bring SONAR online in NRC's secure IT environment including providing access to the secure network for Program delivery staff.

1.2 About the Audit

Objective

The objective of the audit was to provide just-in-time independent assurance to NRC Senior Management that controls remained effective in the Interim Operating Environment (IOE) to allow NRC to make necessary corrections before fiscal year-end.

Scope

The audit scope was defined using a risk-based approach. The audit scope included an assessment of transaction-level transfer payment for IRAP activities based on interim processes and controls in place in FY2015. All programs and initiatives managed under the IRAP umbrella were scoped-in including IRAP firm and organization contributions as well as funding made under the YEP, BIAP, CAIP, and CHTD programs.

The audit focused on contributions awarded after July 28, 2014 and contribution claims processed for the periods spanning June 2014 to March 2015. Risk and control areas that the audit reviewed included project proposal due diligence, contracting authority for contribution agreements, contribution claims management, performance certification (FAA Section 34), financial coding, and records management. The risk assessment excluded the following elements from audit scope; assessments of policies and processes, the management of contribution amendments and Accounts Payable verification activities (FAA Section 33). Elements were excluded due to the expected temporary nature of interim controls pending the implementation of a Secure NRC network and operating systems and the desire to focus on key IRAP related controls due to the inherent risk of transfer payment programs.

OAE randomly selected 40 contribution agreements, for the period spanning September to October 2014, for due diligence and 85 unique contribution claims, for the period spanning June 2014 to March 2015, for review.

Approach and Methodology

The audit was conducted in accordance with generally accepted professional auditing standards of the Institute of Internal Auditors (the IIA) and the standards and requirements set out in the Treasury Board Policy on Internal Audit. The audit criteria, presented in Appendix A, were primarily derived from the TB Policy on Transfer Payments, TBS Audit Criteria Related to the Management Accountability Framework: A Tool for Internal Auditors (2011) and, as applicable, the NRC IRAP Field Manual. Criteria were discussed with senior management in advance of the audit.

The audit addressed the audit criteria as they existed at the time of examination. Audit samples were drawn from across IRAP's operating regions. The audit methodologies were selected to ensure that the root cause of findings was identified and to ensure recommendations add value for NRC. Methodologies are detailed in Table 2 below.

2.0 Audit Findings

2.1 Contribution Eligibility

An effective transfer payments control framework includes clearly defined standards and procedures to assess and demonstrate the eligibility and alignment of contributions to Program objectives. Awarding of contributions must be transparent and properly documented to maintain public trust and adequately demonstrate adherence to program terms and conditions. The audit examined the interim controls put in place to support due diligence and transparency in the assessment of project proposals and the awarding of contributions.

Templates and Tools

We found that IRAP defined and implemented an interim process for proposal due diligence activities complemented by tools and templates to ensure continuity of key controls. Due diligence assessment templates contained the necessary fields to capture the information required for a delegated contracting authority to approve a contribution proposal in a transparent manner. In one instance, we noted that the special condition requirements defined in an interim template was not transferred into the contribution agreement formally approved by NRC and ratified by the recipient. Special conditions are put in place to increase the level of monitoring and oversight of projects based on risk factors determined by the Industrial Technology Advisor (ITA) and or Contracting Authority. The lack of completely defined special conditions increases project risk and reduces the effectiveness of monitoring and oversight.

We also noted instances where the approved contribution agreement had differing template pages. While we did not identify any issues with agreements containing pages from differing templates, it is important that IRAP maintain the integrity of its agreement templates to ensure that awarded contributions contain the most up-to-date terms and conditions and reflect the most up-to-date requirements for all programs managed under the IRAP umbrella. Effective April 1, 2015, IRAP updated all contribution agreement templates and tools which are available through the IRAP intranet and is accessible to all IRAP staff.

Proposal Due Diligence

In 27 of 40 contribution agreements reviewed, we found all necessary information to demonstrate due diligence and eligibility for funding awarded. In 13 of 40 agreements, we noted opportunities to improve records management practices where interim templates did not contain sufficient information to demonstrate project due diligence; all required documentation was later located in regional paper files. It is important that files contain complete information to ensure that all required and necessary documentation is available to support the management of ongoing contributions and to protect the integrity and transparency of the Program. As of May 2015, we noted that IRAP has defined a records management strategy to migrate post-cyber-intrusion recipient files into NRC's secure network. Across a sample of 40 contribution agreements, we found that proposal reviews and assessments generally contained adequate depth and detail of the recipient's business and its plans and strategies; project market opportunities; project technical details; and anticipated project benefits to Canada.

Project Coding

In 40 contribution agreements reviewed, we found that all were coded to the correct financial accounts and Work-Breakdown Structures (WBS) as defined in NRC's financial and project management system to support reporting integrity.

Contribution Agreement Approvals

In 40 contribution agreements, we found that all had the required approvals by a NRC delegated contracting authority and all the agreements were sufficiently supported by ITA technological and business assessments. We noted in 20 of 35 applicable agreements, approval authorities did not provide a brief write-up to justify the contribution award as part of their due diligence activities. Providing a brief description in the interim template "Decision Log" increases transparency of the award process as outlined by the IRAP Field Manual.

Proactive Disclosure

As part of the Government of Canada's Management Improvement Agenda, all contribution agreements over $25K under IRAP and IRAP managed programs are disclosed on NRC's external website. Within our sample of due diligence files, all 22 applicable agreements over $25K were adequately disclosed on NRC's external website with the most up-to-date awarded values and project end dates.

2.2 Contribution Management

An effective management control framework for transfer payments includes clearly defined procedures for management and oversight of contribution agreements, risk-based monitoring activities, clearly defined and properly segregated duties, and a structured claim verification process. The audit examined the interim controls put in place to oversee contribution agreement management including monitoring of project activities and proper approval of contribution claims.

Claim Process

As part of business continuity measures, IRAP implemented a centralized claims management process following the re-initialization of NRC's financial management system in October 2014. All claims were processed centrally in the National Capital Region until a secure network was set-up and secure laptops were made available to regional staff beginning in January 2015. We noted that IRAP reverted to the paper-based management system that was discontinued when SONAR, IRAP's contribution management system, was upgraded in FY2013. Interim tools and templates for claims management and amendments have been put in place and are available through IRAP's intranet.

Claim Review and Approval

We found adequate FAA Section 34 performance certification approval for all 85 claims reviewed. In general, paper-based claims were found to have been adequately completed, reviewed and approved. Within our sample of 85 claims, we noted eight instances where claim calculations were incorrect stemming from mathematical errors in calculating claimed salary costs. Errors were immaterial, ranging from between 0.31% to less than 0.01% of the total value of the contribution agreement where the contribution agreements in question ranged from $20K to $350K in value. We also noted one instance where a contribution to an organization resulted in an overpayment due to the interpretation and calculation of overhead support which was allowable under the terms and conditions of the agreement. The total value of the over-claim did not exceed $100.00 on an agreement totalling $140K.

Across 85 claims reviewed, we found that all were accompanied by a status report to provide an update on project progress and to report on key project information such as projected completion date, project challenges and impediments, anticipated funding burn rates and the receipt of other government funding. We noted two instances where the status report did not meet defined minimum information requirements in relation to report length and one instance where the status report did not disclose any other government funding received within the applicable claim period. All three instances provided an overview of project progress and met all other requirements with exception to the elements described above mitigating the risk of inadequate oversight to monitor project progress. As well, recipients are required to complete a final report that requires disclosure of other government funding received during the life of the contribution agreement.

With regard to claims under the Canada Accelerator and Incubator Program, we noted that in one instance, an advance payment was provided to the recipient and a subsequent advance payment was made before the recipient had provided sufficient claims to off-set the advance payment. Providing advance payments preceding the off-setting of previous advance payments is contrary to the agreement terms and conditions, increases the chance of over-claim, and increases the effort required by Program staff to reconcile and administer claims.

Within a sample of 53 claims processed between August and November 2014, we noted that the median processing time from recipient approval to payment release from NRC's financial system was 12 business days while the average time was 16 business days. Within our sample, the longest period between recipient claim approval and the date NRC had payment ready for release was 65 business days. IRAP has defined a service standard of payment issuance within 35 business days of receiving all properly completed and required documentation. In our review, only 5 of the 53 claims processed between August and November 2014 exceeded 35 business days. Of the five cases, four were related to claims made within the first two months of the cyber-intrusion when interim operating environment controls were being introduced. IRAP was generally able to process claims and continue supporting recipients despite the shutdown of NRC's IT network.

Post-Payment Validation

As part of IRAP's project monitoring activities, post payment validations (PPV)s are undertaken to verify that recipients understand the claiming process, are keeping adequate records of costs incurred, and are settling their liabilities according to the terms and conditions of their contribution agreements. Within our sample of claims, we identified 24 first claims that required PPV activities and only one post-payment validation that had not been completed. We found all necessary documentation to demonstrate IRAP's performance of post-payment validation activities including the provision of timesheets, related payroll documents, and contractor quotes where applicable to demonstrate payment due diligence.

Appendix A: Audit Criteria