Audit of investment planning

NRC has put in place an investment planning framework that is supported by sound governance structures and mechanisms for investment planning that reflect Government of Canada requirements and adequately support the investment planning process within NRC. The agendas and minutes of NRC's investment planning governance bodies demonstrate NRC management's involvement in the oversight of the investment planning process in support of the development and implementation of a departmental investment plan. NRC has provided complementary support and direction through policies, directives and guidance which provide a departmental context to Treasury Board requirements. The governance structure is subject to regular review as evidenced by the adjustments made to the asset management boards in response to changes in the operating environment and revisions to the Terms of Reference for the Senior Executive Committee to reflect their investment planning responsibilities.

No opportunities for improvement were identified.

The roles and responsibilities of committees and boards involved in the investment planning framework are clearly defined, communicated and acknowledged in accordance with their respective mandates, goals and priorities.

Program and Project Services supports the horizontal nature of investment planning in their role as the central coordination body for all investment plan activities above $250K in value. PPS facilitates communication and relationship building between project managers and relevant stakeholders within the organization for procurement, property and asset management, financial advice, project management support and information technology (IT)-related support and guidance. Finance Branch supports project managers in the financial planning and management of projects. Relations with external organizations are managed by NRC's internal common services; Administrative Services and Property Management is responsible for procurement for NRC and has a defined process when interacting with Public Works and Government Services Canada for procurement of goods and services above NRC's authorities. With respect to information technology related projects, roles and responsibilities between NRC and Shared Services Canada are currently being defined and developed.

NRC's reorganization and concurrent changes to the financial and operating model have created challenges to the continued clarity of investment planning roles and responsibilities, particularly as they relate to monitoring the financial progress and performance of the investment plan and individual investment projects. The lack of defined roles and responsibilities for monitoring of financial progress and performance of the investment plan and individual projects has the potential for duplicated efforts, inconsistent reporting to support decision making and an inability to take corrective action in a timely and effective manner.

See Recommendation 3

An opportunity exists to clarify and confirm the roles and responsibilities for financial monitoring of the investment plan between PPS, Finance Branch and portfolio Comptrollers to ensure that funds are effectively and efficiently allocated and that the financial progress of projects is monitored and reported to management in a timely and consistent manner.

As this opportunity is primarily related to monitoring activities, the related audit recommendation is included in Section 5.1(b) of this Table.

2.1 Policies, directives, guidelines and standards on investment planning are identified, available and accessible to employees for reference and use in the course of their investment planning activities. The PPS intranet page provides direct links to Treasury Board investment planning and project management materials as well as related internal guidance such as the NRC Project Management Framework, NRC Project Approval Authority Directive, NRC Project Complexity and Risk Assessment (PCRA) Guide, Guide to Costing of Investment Plan Projects and related tools and templates. Finance Branch provides the Cost Accounting Guide to support consistent costing practices across NRC.

No opportunities for improvement were identified.

2.1a/b/c/d NRC's investment planning processes adequately reflect the requirements of the Treasury Board Policy on Investment Planning – Assets and Acquired Services with regard to a representative and value-added departmental investment plan. Senior management has mandated the adoption of SAP Project System as the departmental project management information system in support of standardized project management methodology and processes and to centrally plan, budget and account for resource allocations.

Following significant changes to NRC's operating and financial models, PPS has been continuously identifying opportunities to improve processes and tools while maintaining their alignment with departmental investment planning objectives. PPS has made incremental changes to tools and templates as opportunities for improvement have been identified; the most recent changes made in Fall 2012 incorporated more detailed instructions with examples to help project managers complete templates with greater efficiency and consistency. Asset lifecycle related information, linkages to NRC's programs, cost and revenue forecasts and the requirement for financial metrics, such as return on investment and payback period, were also incorporated into template updates. Templates are comprehensive and support project managers through the phases of the NRC project lifecycle.

Investment planning processes and activities have been communicated both formally and informally through management communiqués, the PPS intranet and departmental newsletters.

In January 2012, an independent consultant was contracted to undertake an investment planning and project management streamlining exercise to support continuous improvement of processes and practices. PPS is working towards implementing the streamlining exercise recommendations concurrent with the renewal of the departmental investment plan and to align practices with NRC's program-based management model and financial model.

2.2 Investment planning processes were generally found to support appropriate financial stewardship and risk management for consistent decision making.

See 2.2 (a, b and d) for identified opportunities for improvement.

2.2a Investment planning processes are generally linked, in an appropriate manner, to related practices in strategic, business and financial planning and program and project management. Oversight of the linkages is supported by the Asset Management Boards and the Investment Management Committee whom are tasked with ensuring that project submissions are aligned with NRC's Strategy by reviewing all projects in the investment plan, at minimum, on an annual basis. Moreover, processes for corrective action have been defined where non-aligned projects are identified or where new opportunities and requirements emerge requiring reprioritization of investment projects.

NRC's shift to a program-based model occurred following the development of investment planning processes resulting in program and investment planning processes being less than fully aligned. Misalignment of the two processes may result in timing differences between approved programs and investment projects contributing to delays in program activities and potential financial and reputational repercussions where programs are collaborative in nature. Furthermore, program approval does not automatically delegate authority to undertake investment plan projects and delays can be compounded by procurement considerations such as lead time and Contracting Policy requirements. Maximizing the use and value of NRC's investment funding requires that investment planning be coordinated with budgeting activities to ensure proper allocation and reallocation of resources as operating conditions change. PPS is working towards integrating and aligning program and investment planning processes as part of the investment planning streamlining exercise and renewal of the NRC Investment Plan.

Further, the investment planning framework is not supported by a widely accessible database of scientific infrastructure and/or equipment to provide visibility to existing and/or planned resources and investments that would encourage horizontal information sharing and collaboration and could reduce efforts required to develop business cases and plans for already existing or underutilized capabilities. The Scientific Equipment and Labs Asset Management Board is taking steps to address this opportunity and have identified existing capabilities, including functionalities within SAP that can be better leveraged to communicate available equipment and resources across NRC.

See Recommendation 1

An opportunity exists to strengthen the alignment of investment planning activities with program planning and budgeting processes to ensure that NRC resources are efficiently and effectively deployed and redeployed in support of NRC's Strategy. Greater synergies may be realized through strengthened alignment of investment planning tools and templates with NRC's program management model as well as consolidated approvals of programs and related investments. Furthermore, integrating investment planning activities with the overall NRC planning cycle would facilitate the development of a more representative and coordinated financial roadmap to drive decision making.

2.2b/c While costing and budgeting processes generally reflect NRC and Government of Canada requirements, file reviews demonstrated inconsistent methodology in the development of project documentation. Business cases did not consistently assess proposed solutions against the baseline state. Moreover, it was difficult to substantiate that business case and project planning documentation incorporated all relevant NRC policies and practices on budgeting and costing as PPS project planning templates do not require project managers to identify the source information relied on to develop estimates and assumptions.

Two separate documents, the Cost Accounting Guide and the Guide to Costing of Investment Plan Projects, have been developed by Finance Branch and PPS which establish NRC costing standards and practices. The Guide to Costing Investment Plan Projects focuses directly on investment planning related costing practices, in greater depth and detail than the Finance Branch Cost Accounting Guide. The Cost Accounting Guide provides a holistic view of project costing from a program-model perspective. There are opportunities to better align the two documents for greater consistency and to realize potential synergies between PPS and Finance Branch in supporting NRC's program and project management communities.

PPS tools and templates require the identification of project risks and their incorporation in project planning documentation with appropriate mitigation plans. The tools and templates incorporate elements of NRC's risk management methodology; PPS should consider updating the tools and templates when NRC's updated risk management framework is approved by management. The majority of projects reviewed budgeted a contingency that was in line with costing guidance.

In the development of project management plans, project managers were inconsistent in their risk management practices with risk registers focusing on procurement related risks and less emphasis on risks related to adequate staffing and resources to execute the project, integration of new and old systems, training and issues related to set-up and start-up and interruption to operations among others. Inadequate and inconsistent risk management practices may adversely impact the execution of investment projects resulting in delays and cost overruns. As well, proper risk management may enable project managers to recognize and act on positive risks.

See Recommendation 2

The opportunities identified are primarily related to training and not process weaknesses with resulting recommendations included in Section 3.1 of this Table.

Areas for improvement were identified with regard to:

• Consistent comparison of proposed solutions with the baseline state in business cases to demonstrate the value of the project as well as implications of not undertaking any action;
• Identification of information sources used to generate key underlying assumptions and estimates in the project business case including assumptions for ongoing maintenance and lifecycle related costs to address Treasury Board's whole-of-life cost concept; and
• Consistent risk management practices and methodology to guide decision making, support prioritization of actions, improve understanding of threats and opportunities and effectively allocate resources.

2.2d Formal and informal continuous processes are in place to challenge assumptions and variances identified in the investment planning process at different levels in the governance structure for investment planning. Oversight structures and mechanisms are in place to challenge the basic, underlying assumptions of project submissions, plans, budgets and estimates.

While the role and responsibility of Finance Branch Comptrollers for monitoring the financial progress of capital investments within their respective portfolios is understood, no process has been defined to communicate and resolve identified issues and variances with related stakeholders including PPS and portfolio management. The lack of a defined communication process between FB Comptrollers and key stakeholders increases the risk of duplicated efforts by different monitoring bodies, inconsistent information for decision making and unsatisfactory resolution of identified issues.

See Recommendation 3

This issue relates primarily to monitoring and is addressed by the audit recommendation included in Section 5.1 of this Table.

2.2e Projects are being adequately assessed for risk and complexity according to their scope and objectives through the use of the Treasury Board Project Complexity and Risk Assessment Tool. PPS has also developed a NRC Project Complexity and Risk Assessment Guide tailored to support NRC project managers in the use of the tool.

No opportunities for improvement were identified.

Recommendation 1: PPS should continue working toward integrating investment planning processes, program planning activities and NRC's planning cycle and continue to support initiatives to leverage tools and capabilities that will maximize the use of existing resources prior to consideration of new and/or additional investments. [Priority: Moderate]

3.1 Formal training is provided to employees involved in the investment planning process and related activities to support the development of investment planning competencies. At the inception of the investment planning framework, PPS provided comprehensive training including instruction in the use of NRC's approved tools and templates and project management methodology. PPS has also provided training to project managers with focus on project planning and project management within SAP Project System. PPS has noted its intention to develop competency profiles for project managers to foster consistency in knowledge and practices within the community.

See Recommendation 2

With PPS plans to develop training and competency targets for investment planning, there is an opportunity to establish a timeframe to address investment planning competencies via project manager competency profiles. Competency profiles ensure consistency in development and knowledge as well as facilitating easier identification of training requirements and succession planning.

Furthermore, there are opportunities to focus training by:

• Providing targeted training in costing and financial management competencies and performance management (e.g. focused on quantitative performance metrics, such as return on investment) and development of value statements to demonstrate financial stewardship and performance measurement.
• As noted in 2.2(b), while there is evidence of improvement in more recent investment project files, there were still documentation weaknesses that can be addressed through improved training and monitoring.

3.1a/b Training and materials related to the Project Management Framework and methodology are generally adequate to support employees in delivering on their investment planning responsibilities. Given the changes to the operating environment surrounding investment planning, training on the use of investment planning templates and tools were targeted at project managers with approved projects only. General training sessions were discontinued until investment planning processes and activities are streamlined and integrated with NRC planning and program activities as part of the renewal of the departmental investment plan.

Systems related training has focused on the use of SAP Project System. Project managers and employees have been provided general training on financial management principles and concepts as part of SAP Project System training to create, manage, report on and monitor projects. Finance officers and individuals assigned to the Finance Branch Costing Group have received targeted training for Project Accounting within Project System.

3.1c PPS facilitates ongoing development and continuous improvement of investment planning skills and knowledge with standardized templates and a project close-out process that requires project managers to document their lessons learned and best practices to be shared with the general investment planning and project management community. Microsoft SharePoint is leveraged as a collaborative resource, serving as a forum for sharing information between PPS and project managers. Project managers noted inconsistent feedback from PPS and management in relation to submitted investment plan submissions. The lack of feedback precludes project managers from tailoring their level of effort to investment planning activities and developing a general understanding of requirements and expectations from PPS and the investment planning framework.

A Project Management User Community was recently created in an effort to facilitate knowledge sharing, development of organizational expertise as well as bi-directional feedback between project managers and the NRC project management authority.

No opportunities for improvement were identified

3.2 NRC leverages SAP Project System to support managers in the identification of resources, their availability, commitments and utilization levels reflecting NRC's new operating and accounting models. Training activities are on-going to develop the necessary competencies, knowledge and skills set across NRC in basic project management and use of SAP Project System.

No opportunities for improvement were identified.

Recommendation 2: PPS should consider establishing a timeline for developing and implementing training and competency targets for Project Managers. These targets should address specific areas including costing, development of assumptions, project management, performance management (including use of key performance measures such as ROI), financial management concepts, budgeting, and documentation of investment project files. [Priority: Moderate]

4.1 Information is readily available and sufficient for decision making purposes for management at varying levels within the organization. NRC's adoption of SAP Project System, implementation of business intelligence software and internal initiatives to develop workflow procedures to manage investment planning documentation facilitates information access and dissemination to support rational resource allocation and oversight of investment planning activities.

No opportunities for improvement were identified.

4.1 a/b/c/d Information provided to decision makers for investment planning purposes is sufficient, timely and reliable. Project managers are also required to provide semi-annual status reports to PPS to communicate changes, emerging risks and general progress in a structured manner. PPS also provides financial progress reports detailing spending against the budgeted investment plan for major capital spending (projects above $250K in value only) to the Senior Executive Committee. Finance Branch introduced a new financial monitoring process for branch and portfolio comptrollers to provide monthly financial progress reports, which integrate investment plan projects, to General Managers and Directors General to support management monitoring and oversight. NRC has recently adopted quarterly performance reviews that incorporate analysis of investment plan progress to keep management informed and reactive to changing conditions.

SAP Project System, Business Explorer and related business intelligence software support capturing of, analysis and reporting on relevant information for decision making purposes related to investment planning activities. The enterprise wide adoption of SAP Project System has facilitated collaboration and flexibility, enabling project managers to oversee resources across NRC contributing to individual projects in a structured and consistent manner. Integration of Project System with the financial management, controlling, sales and distribution, human resources and material management modules within SAP provides NRC with a holistic view of NRC's activities facilitating more efficient and effective resource allocation, analysis of cost drivers and areas for cost reduction, and monitoring of revenue.

Collection and measurement of performance indicators for investment projects including financial metrics is enabled through SAP Project System. Project managers are provided the ability to organize projects by work breakdown structures with specific financial targets for defined activities. PPS also provides performance reports to the Senior Executive Committee on a monthly basis reporting on the elements of time, budget and scope through a manual project tracking process.

5.1 NRC has the necessary elements to support a monitoring framework to ensure effectiveness and compliance of NRC investment planning activities with applicable Government of Canada and NRC policies, directives and processes. Monitoring and oversight activities are occurring at different levels of management to provide the Deputy Head with assurance that the NRC investment planning process is effective, efficient and demonstrates value and stewardship of public funds. As noted in 1.2, there is a need to better define the body responsible for monitoring the investment planning framework and process within NRC to ensure its effectiveness and continued compliance with Treasury Board requirements.

See Recommendation 3

Related opportunities for improvement are provided in connection with audit criteria 1.2.

There are opportunities to better ensure the completeness of projects included within the investment planning process, the consistency of practices related to all investing activities and the effective monitoring of investment planning, at both the tactical and strategic level by:

• Implementing a monitoring process to ensure that all real property, IT, scientific equipment and strategic enabler-type projects that meet the risk and complexity criteria of an investment plan project are captured within the investment planning framework and subject to the rigor and oversight of the framework; and
• Assigning responsibility to help ensure that investments falling outside of the investment planning process (e.g. projects under $250,000 in value) are managed by respective Divisions according to their risk and complexity and in a manner that is consistent across NRC.
• Integrating the monitoring elements to create a comprehensive monitoring framework to support investment planning within NRC. Assigning responsibility to measure the performance of the departmental investment plan as a whole by integrating the monitoring activities of PPS (major capital activities), Division Heads (minor capital activities) and Finance Branch (financial activities) would provide management with a holistic assessment of NRC's effectiveness and compliance in the allocation and reallocation of funding to achieve departmental strategic outcomes in accordance with Government of Canada and NRC policies, directives and processes.

5.1a/b/c The effectiveness of the investment planning framework was reviewed by an external consultant to identify opportunities for streamlining and simplification in January 2011. Implementation of the streamlining exercise recommendations is ongoing as PPS seeks to align the investment planning and project management framework with NRC's program-based management model and related financial and operational processes and activities. Other priorities and changes to NRC's operating environment have delayed the development of a schedule for third party reviews of the investment planning and project management framework to support continuous improvement.

Project progress is monitored by the Senior Executive Committee through monthly project status and financial reporting, and quarterly budgeting and performance reviews. Finance Branch Comptrollers have been tasked with providing financial analysis and monitoring of investment projects within their branches and portfolios as part of recently adopted financial oversight practices.

There is an opportunity to ensure risks continue to be effectively managed by clarifying and formalizing the roles and responsibilities among monitoring bodies to ensure a coordinated approach between PPS and Finance Branch to avoid duplication of efforts and inconsistent reporting, escalation and resolution of identified issues.

Investment planning monitoring and oversight is performed at varying levels of the organization. Projects over $250,000 in value are managed according to the NRC Project Management Framework and NRC investment planning process. Projects under the threshold are managed at the Division level by respective Division Heads. PPS has no direct responsibility for oversight of minor capital investments.

Following changes to the operating environment, responsibility and accountability for ensuring that the NRC Investment Plan incorporates all investments of commensurate risk and complexity that align with the definition of a project within the NRC Project Management Framework is unclear. Ensuring that no investments bypass or are excluded from the investment planning process demonstrates an effective investment planning process and framework and may have implications on the NRC OPMCA rating and consequently, departmental delegated authorities from Treasury Board. As well, NRC currently lacks a body mandated with responsibility to verify that activities undertaken deliver on approved business cases and plans with concrete deliverables and quantifiable results.

5.1d PPS collects information on best practices and lessons learned from completed projects, incorporating them into continuous improvement efforts on templates, tools and guides. The recent creation of a Project Management User Community for bi-directional feedback between the NRC project management authority and project managers represents a new forum for communication and knowledge sharing. SharePoint is also leveraged as a resource to convey best practices and lessons learned to the overall community.

No opportunities for improvement were identified.

5.2 On an annual basis, NRC reviews the alignment of its funded and unfunded investment plan projects against the department's strategic direction; projects deemed to not directly contribute to the achievement of NRC's Strategy are removed and the remaining projects are reprioritized. Investment plan projects overseen by PPS are reviewed on a periodic basis to assess their progress against the approved budget, schedule and scope and corrective action is taken where necessary.

Management efforts to align the investment planning, program management and budgeting processes would facilitate the review of investment and project alignment with NRC's strategic direction; the approval or termination of a research program would have direct implications on associated investment projects and facilitate decision making in the allocation or reallocation of funds to projects of greater strategic priority.

See Recommendation 3

Related opportunities for improvement are provided in connection with audit criteria 1.2 and 5.1(c).

The framework for monitoring the performance of investments is an area where a number of opportunities were identified:

• Improving guidance and practices to demonstrate the link between investment projects and NRC's overall objectives;
• Improving the monitoring of quantifiable performance metrics for investment projects (with a view to improving comparability and objectivity of investment decisions);
• Implementing a framework to support the measurement of project outcomes with objectives and assertions identified in the project business case and project management plan;
• Implementing a framework that measures the performance of NRC's investment planning activities holistically, including all elements of major and minor capital and acquired services.

5.2a Project performance characteristics such as budget, schedule and scope are measured on a periodic basis and monitored by the Senior Executive Committee. Performance measurement efforts have improved with updates to investment planning templates to ensure consistency in the use of quantitative metrics for investment decisions.

While PPS and Division Heads are responsible for managing their respective elements of the departmental investment plan, no single entity has been mandated with measuring the performance of the NRC Investment Plan as a whole. The lack of performance measurement of the departmental investment plan as a whole precludes NRC's ability to assess departmental performance in the allocation and reallocation of resources to new and existing assets and acquired services to achieve NRC's desired strategic outcomes.

Recommendation 3: Integral to its mandate to manage and oversee the departmental investment plan, PPS should consider implementing a comprehensive monitoring framework to measure overall performance of the investment plan and NRC's performance as a department in the allocation and reallocation of resources. The framework should incorporate all elements of investment planning managed at the PPS and Division Head Levels, including acquired services, and should include quantitative performance metrics to ensure continuous improvement. [Priority: Moderate]

1. An effective governance structure and related governance mechanisms are in place.
   1. Management is actively involved in the oversight of the investment planning process to support the development of a departmental investment plan.
   2. The governance structure and mechanisms reflect Government of Canada accepted practices and support alignment of investments with NRC strategy
   3. The governance structure is regularly reviewed for appropriateness in light of changes within, or external to, NRC

1. Roles and responsibilities, including those of external parties, are clearly defined and effectively communicated, understood and acknowledged.
   1. Roles and responsibilities support the horizontal nature of the investment plan and supports allocation of resources in a strategic manner (supports the overall organizational strategy, direction and vision)

1. There are clearly communicated policies, directives, guidelines and standards which define investment planning processes across NRC.
   1. NRC investment planning processes reflect Treasury Board requirements and/or guidance and support NRC's OPMCA rating
   2. Processes and related tools are complete, aligned and support NRC's investment planning objectives
   3. Investment planning policies, directives and standards are communicated to, understood and acknowledged by NRC staff
   4. Process and other changes resulting from the streamlining initiative by Program and Project Services support NRC's investment planning framework and the associated recommendations are progressing as per approved management action plans

1. NRC's investment planning processes support rational, informed and supported financial stewardship and risk management through a standardized and methodical approach to investment planning.
   1. Investment planning processes are appropriately linked with, and aligned to, related processes such as; costing, budgeting and forecasting; strategic, business and financial planning; and program and project management
   2. Investment costing and budgeting processes account for total project costs and reflect: analysis of investment options, risk information/contingencies, lifecycle management requirements and NRC/Treasury Board guidance
   3. Planning/budgets incorporate risk mitigation plans
   4. There is a formal, continuous and structured review and challenge of assumptions and variances
   5. Project complexity and risk assessments (PCRA) are effectively and appropriately used

1. Formal training is provided to appropriate employees in support of developing relevant investment planning competencies.
   1. Training and materials support project managers and employees on key investment planning processes, systems and related tools
   2. Project managers and employees are provided with appropriate training in financial management principles and concepts to effectively develop budgets, forecasts and informed assumptions
   3. A continuous improvement program supports the ongoing development of investment planning skills and knowledge (e.g. Lessons learned and project results are communicated across the organization to share best practices)

1. There is a process to support Managers in identifying the availability of resources reflecting the new departmental accounting model, new information technology resources and processes and cross-government fiscal restraint.

1. Information and systems are available to support decision-making across NRC.
   1. Information provided to decision-makers regarding investment projects is sufficient, reliable and timely
   2. Systems support the effective capture, synthesis, analysis and reporting of information.
   3. Systems enable collaboration, flexibility and responsiveness given the departmental focus on revenue generation.
   4. Systems track and collect performance information of projects including financial metrics

1. A monitoring framework is in place and monitoring activities are undertaken to ensure effectiveness and compliance with applicable Government of Canada and NRC policies, directives and processes.
   1. There is a process for monitoring the effectiveness of the investment planning framework
   2. Project progress is monitored and corrective action is escalated/taken until issues are resolved or the project is terminated
   3. There is a process to ensure all investment projects are identified and tracked (e.g. split projects/ funded projects, and under-costed projects are identified)
   4. Monitoring supports the continuous improvement of investment planning practices and processes

1. Investments and projects are reviewed on a periodic basis to monitor progress and reassess alignment with NRC's Strategy.
   1. The performance of investments is monitored through an effective framework to define, collect, measure, assess and report the effectiveness of investment decisions (e.g. performance against those metrics which are aligned with NRC's investment objectives)