Evaluation of the Herzberg Astronomy and Astrophysics Research Centre

Approach

This evaluation used a mixed‑methods approach, incorporating both qualitative and quantitative data from several lines of evidence. This allowed for triangulation (i.e., convergence of results across lines of evidence) and complementarity (i.e., developing better understanding by exploring different facets of a complex issue) of the evaluation findings. In addition, a Gender‑Based Analysis Plus lens was applied throughout the evaluation.

Methods

The evaluation included:

• document review
• data review (administrative and performance data)
• survey of Canadian astronomers and astrophysicists (n=133, 17% response rate)
• internal focus groups and interviews (n=28)
• external interviews (n=16)
• peer review with national and international experts (n=6)

Evaluation questions

Scientific Excellence

1. To what extent is the Herzberg Astronomy and Astrophysics Research Centre a leader in scientific excellence?

Relevance

2. Is the Herzberg Astronomy and Astrophysics Research Centre focussed on the right areas?

Engagement

3. Has the Herzberg Astronomy and Astrophysics Research Centre engaged with the most appropriate partners and collaborators in the most effective ways?

Capabilities

4. To what extent does the Herzberg Astronomy and Astrophysics Research Centre have the capacities, competencies and facilities to achieve its objectives?

Summary of Expected Results

5. To what extent has the Herzberg Astronomy and Astrophysics Research Centre achieved or contributed to its expected results?

For more detailed information on the methods, including limitations, refer to appendix A. Short biographies of each peer review member are located in appendix B.

Guiding principles

• The National Research Council Act
• International Contributions Program: the international agreements carry obligations to ensure that observatory use is merit‑based and commit the partners to maintain the facilities at competitive levels
• Long Range Plan (LRP): the plan provides the context for operational decisions, prioritization of projects, deployment of resources, development of new competencies and capital investments
• University community engagement: engagement with various university and research networks, such as the Association of Canadian Universities for Research in Astronomy (ACURA), to provide continuous external validation of the Herzberg Astronomy and Astrophysics Research Centre's internal work ensuring its relevance in supporting the astronomy community in achieving its goals

Approach

• Observatory support: engages in governance, manages observatory access, provides expert assistance, operates Canadian Astronomy Data Centre (CADC)
• Research community support: provides on‑site observatory staff and distributed operations for observatories (including maintaining a unique site for community radio telescopes)
• National and international scientific and engineering leadership: enables large‑scale science and instrumentation development projects as well as adjunct positions within the Herzberg Astronomy and Astrophysics Research Centre
• The Herzberg Astronomy and Astrophysics Advisory Board: provides independent strategic advice to the Herzberg Astronomy and Astrophysics Research Centre from diverse perspectives, representing the range of the astronomy community (government, academia, research and industry)

See appendix C for the Herzberg Astronomy and Astrophysics Research Centre's logic model.

Most revenues generated by strategic research

• Over the evaluation period, the Herzberg Astronomy and Astrophysics Research Centre generated $11.1M in revenues and $242M in expenses.
• The nature of the NRC is to make investments into science. Revenues are not intended to exceed expenses.
• Strategic research revenue accounted for 86% ($9.5M) of the research centre's total revenue over the evaluation period, while technical service revenues accounted for 12% ($1.3M). Other revenue sources (2% of total over the evaluation period, $274K) came from royalties and licenses, lease and use of property, sales of goods and information and products, and grants and contributions.
• It is important to note that the research centre's revenues are not linear and are based on a few large‑scale international projects with partner organizations. The research centre does not have the ability to adjust revenue targets on a year‑by‑year basis.

 

Figure 2 – Text version

A bar graph depicting the total revenue of the Herzberg Astronomy and Astrophysics Research Centre in millions of dollars by fiscal year. Five fiscal years were examined, between 2016‑17 and 2020‑21. A line is superimposed on the bars to indicate Herzberg Astronomy and Astrophysics Research Centre's target revenue.

The data shows that total revenue was below the revenue target from revenue from 2016-17 to 2018-19 was below revenue targets and exceeded revenue targets from 2019-20 to 2020-21.

• For 2016‑17, the earned revenue was approximately $2.9 million and was below the target revenue.
• For 2017‑18, the earned revenue was approximately $1.4 million and was below the target revenue.
• For 2018‑19, the earned revenue was approximately $1.8 million and was below the target revenue.
• For 2019‑20, the earned revenue was approximately $2.5 million and was above the target revenue.
• For 2020‑21, the earned revenue was approximately $2.5 million and was above the target revenue.

 

 

Infographic – Text version

Infographic that displays the total expenses yearly average $48.4 million as well as the lowest year of expenses in 2017‑18 $44.3 million, and the highest year of expenses in 2019‑20 $52.8 million.

 

Workforce

As of March 31, 2021, the Herzberg Astronomy and Astrophysics Research Centre had a total of 149 staff including 89 technical and scientific researchers, 11 technical officers, 25 students (both graduate and co‑op), 21 administrative/corporate staff and 5 management staff (includes Directors and the Director General).

In addition, there are 14 Common Services FTEs that directly support the Herzberg Astronomy and Astrophysics Research Centre (report centrally, but managed by the Herzberg Astronomy and Astrophysics Research Centre Operations Director).

Gender, diversity, and inclusion

When compared to labour market availability, the Herzberg Astronomy and Astrophysics Research Centre has good representation of women in administrative, professional as well as semi‑professional and technical positions. There were no women, however, in the 5 middle (Directors and Managers) or 1 senior management (Director General) positions.

As is the case across the NRC's research centres, visible minorities, Aboriginal people and persons with disabilities are all underrepresented in the Herzberg Astronomy and Astrophysics Research Centre's occupational groups. As there are fewer than 5 Aboriginal people and persons with disabilities at the Herzberg Astronomy and Astrophysics Research Centre's the workforce availability data were unavailable for these groups.

 

Figure 3 – Text version

A vertical clustered bar chart that depicts the workforce representation of women at the Herzberg Astronomy and Astrophysics Research Centre compared to labour market availability in the positions of administrative, professional, semi‑professional, middle managers and senior managers.

The data show that women's representation at the Herzberg Astronomy and Astrophysics Research Centre was above labour market availability in administrative, professional and semi‑professional positions but was below labour market availability in middle and senior manager positions.

• Women's representation in administrative positions was 100% and exceeded labour market availability of 83%.
• Women's representation in professional positions was 19% and exceeded labour market availability of 17%.
• Women's representation in semi‑professional positions was 15% and exceeded labour market availability of 12%.
• Women's representation in middle manager positions was 0% and was below labour market availability of 43%.
• Women's representation in senior manager positions was 0% and was below labour market availability of 28%.

 

Other Government Departments

• 5 agreements with Canadian Federal Government entities
• Total revenue of agreements is $0.8M
• Responsible for 10% of total revenue

Academia & Others^{Footnote 1}

• 25 agreements
• 20% Canadian and 80% international
• Total revenue of agreements is $4.4M
• Responsible for 52% of total revenue

Industry

• 15 agreements
• 40% Canadian and 60% international
• Total revenue of agreements is $3.1M
• Responsible for 38% of total revenue

Optical Astronomy Directorate (OAD)

• Operates at the Dominion Astrophysical Observatory (DAO) site and includes the Canadian Astronomy Data Centre
• Supports and facilitates access to optical, ultraviolet and infrared telescopes for Canadian astronomers
• Provides in‑kind and contracted support for the operation of Canada's UV/optical/IR observatories, and acts as the interface between these observatories and the Canadian astronomy community
• Operates and manages the onsite 1.8‑m Plaskett and 1.2‑m telescopes

Radio Astronomy Directorate (RAD)

• Operates at both the DAO and the Dominion Radio Astrophysical Observatory (DRAO) sites
• Supports and facilitates access to millimeter/submillimeter and centimeter observatories
• Acts as the interface between the Canadian astronomy community and the Atacama Large Millimetre/submilimeter Array (ALMA), and provides scientific and technical services to support Canada's national and international radio telescopes
• Supports the hosting of University owned telescope experiments at the DRAO

Astronomy Technology Directorate (ATD)

• Operates at both DAO and DRAO sites
• Designs and develops instruments and observatory infrastructure for the national and international telescope facilities supported by Canada
• Provides scientific, technical and advisory services

Canadian facilities

• Dominion Astrophysical Observatory (DAO)
• Dominion Radio Astrophysical Observatory (DRAO)

International facilities

• Atacama Large Millimeter / submillimeter Array (ALMA)
• Canada‑France‑Hawaii Telescope (CFHT)
• Gemini Observatory

Future facilities

• Square Kilometre Array (SKA)
• Thirty Meter Telescope (TMT)

Key examples of the Herzberg Astronomy and Astrophysics Research Centre scientific achievements

Over the evaluation period, the research centre's contributions have led to:

• development of new instrumentation, including: the Square Kilometre Array (SKA) Correlator and Beam Former, the Thirty Meter Telescope (TMT), Narrow Field InfraRed Adaptive Optic System (NFIRAOS), Gemini Infrared Multi‑Object Spectrograph (GIRMOS) and Gemini Planet Imager (GPI)
• new scientific discoveries relating to priority research questions in astronomy (e.g., astrometric reference frame for the NASA New Horizons mission)
• industry applications (e.g., Calian Advanced Technologies built a dedicated facility in Saskatoon, Canada to produce a new line of 6‑m and 10‑m composite dish antennas for satellite communications at higher frequencies based on Herzberg Astronomy and Astrophysics SKA‑related technology)

The Herzberg Astronomy and Astrophysics Peer Review Committee (PRC) found that for a small organization, the Herzberg Astronomy and Astrophysics Research Centre manages an impressive number of research and technical projects and is providing high value to Canadian astronomers.

Use of Herzberg Astronomy and Astrophysics developed instrumentation is an indication of scientific excellence

Between 2012‑2017, the research centre contributed to the design and construction of 2 advanced high‑resolution spectrographs (SPIROU at CFHT and NIRPS at ESO/La Silla) that conduct large, dedicated surveys for rocky planets around hundreds of M‑type stars.

Major Herzberg Astronomy and Astrophysics instruments operating at large observatories are heavily used to deliver high‑impact science. These include the WIDAR correlator for the Jansky Very Large Array (JVLA), the ALMA Band 3 receivers and the Gemini Multi‑Object Spectrographs (GMOS).

50% of the observing time at Gemini uses GMOS even though these twin spectrographs were commissioned nearly 20 years ago.

The research center's support for Canadian Hydrogen Intensity Mapping Experiment (CHIME) in terms of site location, postdoctoral fellows (PDFs) and staff will contribute to the detection and localization of nearly all CHIME‑detected Fast Radio Bursts (FRBs).

"The Herzberg Astronomy and Astrophysics Research Centre has the ability to punch above its weight continually on the international stage and to support the universities in their pursuits of new missions such as CHIME."

The Herzberg Astronomy and Astrophysics Research Centre's involvement at astronomy gatherings

From 2016 to present, Herzberg Astronomy and Astrophysics staff participated in 66 committees (49 international), 12 international working groups and 11 advisory boards.

Over the same time period, staff attended 54 national and international astronomy conferences and made presentations at 39 national and international institutes, conferences and workshops.

Research centre staff are recognized, by stakeholders, for their ability to bring parties together and to help address sensitive issues, such as those around Indigenous engagement or equity, diversity and inclusion (EDI) goals.

The Herzberg Astronomy and Astrophysics Research Centre awarded nationally and internationally

Over the evaluation period, research centre staff were recognized for their contributions to research through national and international awards, including:

• Governor General's Innovation Award (2020)
• Guggenheim Award (2020)
• Fellow into Royal Society of Canada (2020)
• Breakthrough Initiative's New Horizons in Physics Prize (2020)
• American Astronomical Society's Warner and Rubin Prizes (2019)
• Gruber Prize for cosmology (2018)
• Natural Sciences and Engineering Research Council Herzberg Gold Medal (2016)

The Herzberg Astronomy and Astrophysics Research Centre successfully publishes its research

Over the evaluation period, the research centre published 960 unique peer‑reviewed publications, exceeding their 5‑year target of 600.

Citation scores varied between 1.72 and 2.36, also exceeding the annual NRC average citation score of 1.5.

The research centre employs highly regarded scientists who are actively advancing astronomical science and publishing their findings on a wide variety of topics.

The PRC noted that the research centre is publishing in high‑impact scientific journals, such as nature and has produced highly regarded engineering papers.

Since the 2016 evaluation, the research centre's access to domestic telescopes resulted in 49 refereed publications and access to international telescopes led to 796 refereed publications.

Key numbers coming out of the CADC

The data from the CADC are seen as highly relevant for the astronomy community; 723 refereed publications have acknowledged the use of CADC data over the evaluation period.

In addition, over the past 5 years, membership registration for the CADC has increased from 4,498 to 8,000.

 

Figure 5 – Text version

A vertical bar and line chart that depicts the number of Herzberg Astronomy and Astrophysics Research Centre's publications vs. targets over the evaluation period (2016‑2021)

The data show that the Herzberg Astronomy and Astrophysics Research Centre publication numbers constantly exceeded the yearly target from 2016‑17 to 2021‑22.

• In 2016‑17 the research center published 210 unique peer‑reviewed publications
• In 2017‑18 the research center published 163 unique peer‑reviewed publications
• In 2018‑19 the research center published 216 unique peer‑reviewed publications
• In 2019‑20 the research center published 156 unique peer‑reviewed publications
• In 2020‑21 the research center published 215 unique peer‑reviewed publications

 

The Research Centre's engagement in international and domestic facilities indicates relevance

The PRC found that the Herzberg Astronomy and Astrophysics Research Centre's activities and engagement in international research facilities, such as ALMA, Jansky Very Large Array (JVLA), SKA, Next Generation Very Large Array (ngVLA), TMT, Gemini and CFHT, as well as new domestic facilities such as CHIME and CHORD are indicators of its strong focus and relevance.

Survey respondents who engage in ground‑based observational research use a wide range of ground‑based telescopes, including but not limited to those supported by Canada.

• 84% of survey respondents that engage in ground‑based observational research requiring access to telescopes, reported that the international telescopes supported by Canada are critical to their research needs.
• 32% of the same respondents reported that the domestic ground based telescopes (located at the DAO and the DRAO) were critical to their research.

Both international and domestic telescopes supported by Canada are oversubscribed by astronomers, demonstrating their relevance.

The Herzberg Astronomy and Astrophysics Advisory Board

The Herzberg Astronomy and Astrophysics Advisory Board was implemented in response to the 2016 evaluation recommendation to establish a formal consultation process to allow the Canadian astronomy community to provide strategic advice on the research centre's scientific activities and priorities.

The Advisory Board provides diverse perspectives representing the range of the astronomy community (10 members from: government, academia, research and industry).

The Advisory Board provides strategic recommendations to the research centre (e.g., the development of a formal internal process to prioritize the Herzberg Astronomy and Astrophysics Research Centre's participation in projects).

Among external stakeholders from the Canadian astronomy community, there is limited awareness of the existence of the board and its purpose. As a result, there may be a risk that the research centre has not included the broader perspective of Canadian astronomy community in their activities and priorities.

Aligns with the strategic objectives of the federal government

The Herzberg Astronomy and Astrophysics Research Centre's current and future objectives are aligned with federal government objectives of pursuing fundamental research (Budgets 2019 & 2021) and ensuring pure science is available to researchers and the public through the CADC (2019 NRC President's Mandate Letter).

Canada's Science Vision identifies the goals of fostering the next generation of scientists and making Canadian science more collaborative. The research centre meets these goals by training students and Postdoctoral Fellows (PDFs), and through its partnerships with universities, institutes and industry on research projects.

Aligns with the NRC's mandate and strategic areas of focus

The research centre fulfills the NRC's mandate "to operate and administer any astronomical observatories established or maintained by the Government of Canada."

The Herzberg Astronomy and Astrophysics Research Centre supports the NRC's strategic area of focus of understanding our world through fundamental scientific research by contributing to large astronomical projects and the advancement of astronomy.

The Herzberg Astronomy and Astrophysics Research Centre also supports the NRC's strategic area of focus of creating Canadian wealth through innovation by employing Canadian small and mid‑size enterprises (SMEs) to supply technical solutions for observatories and by engaging with SMEs to apply the Herzberg Astronomy and Astrophysics Research Centre developed astronomical technologies to industry, such as the prototype dishes developed for SKA now used by industry in Saskatchewan, Canada.

Canadian and international collaborations and partnerships

Most of the Herzberg Astronomy and Astrophysics Research Centre's activities include partners and collaborators which allow for knowledge transfer, access to specialized skill sets and assistance in managing complex projects. The research centre relies on these collaborations to increase efficiency.

Over the evaluation period, the Herzberg Astronomy and Astrophysics Research Centre engaged with Canadian (e.g., Calian Advanced Technologies in Saskatoon, Canada) and international (e.g., EMSS in South Africa) industry partners, government departments (e.g., Canadian Space Agency ‑ CSA), and university and research networks (e.g., the International Association of Universities for Research Astronomy).

Some of these relationships are contracting‑out agreements, with elements of collaboration, knowledge transfer and capacity building.

The PRC found the research centre has a good track record in terms of commercial engagement, including commercial spin‑off of Intellectual Property (IP) and other technical developments to industry.

In addition, the PRC reported that the research centre has energetic engagements that led to the development of new ideas such as the astrophotonics project (briefly described in the Collaboration with other NRC research centres section).

Strategic partnerships

Over the evaluation period, the Herzberg Astronomy and Astrophysics Research Centre signed 45 agreements with Canadian and international partners with a total revenue of $8.3M.

The research centre has engaged with major telescope organizations (e.g., ALMA, Gemini, SKA, TMT) to allow the Canadian astronomy community access to leading facilities.

The PRC indicated that through participation in major international and domestic astronomy projects, the Herzberg Astronomy and Astrophysics Research Centre demonstrates effective engagement with appropriate partners and collaborators, including some strategic partnership arrangements with industry.

The PRC remarked that the research centre tend to partner early with external organizations to gain specific expertise (e.g., staff became engaged in TMT management to increase complex project management skills) which increases internal capacity and reduces project risks.

The Herzberg Astronomy and Astrophysics Research Centre's expertise and strong project management have led to a solid reputation within the astronomy community and have strengthened its ability to attract national and international partners.

 

Figure 6 – Text version

A horizontal line chart that depicts the total revenues for partnership agreement by fiscal year.

In 2015‑16 the total revenue was $2,707,190, it decreased in 2016‑17 to $1,089,287 further decreased in 2017‑18 to $546,739 and decreased to the lowest point in 2018‑19 at $106,257. In 2019‑20 it increased to $1,506,345 and in 2020‑21 total revenue reached a high of $2,367,154.

Table notes

Table note *

2015‑2016 data were included as some revenues associated with that fiscal year's agreements were received during the evaluation period.

Return to table note * referrer

 

"The Herzberg Astronomy and Astrophysics Research Centre brings in external assistance early to gain specific expertise to augment their technical talent and to mitigate risk."

Collaborations with other NRC research centres

Within the NRC, the Herzberg Astronomy and Astrophysics Research Centre has evolved from working in isolation, as observed in the 2016 evaluation, to developing collaborations with other NRC research centres, including the Nanotechnology Research Centre (e.g., lower‑voltage deformable mirrors for image quality for Finding Earth 2.0) and the Advanced Electronics and Photonics Research Centre (e.g., astrophotonic chip development for instrumentation to discover life outside our solar system).

There may be opportunities for the Herzberg Astronomy and Astrophysics Research Centre to seek further collaborations with other NRC research centres.

Opportunities to increase awareness

While the Herzberg Astronomy and Astrophysics Research Centre is well known in the Canadian astronomy community, there is less awareness within some segments of the international astronomy community, industry and among the public.

The PRC suggested that the research centre improve communication of its outcomes and successes to both external stakeholders and the public.

In addition, the PRC felt that the research centre could improve its strategies for connecting to industry by taking a more targeted approach such as exploring opportunities for industry training and internships to enhance staff capacity and skills in‑line with the Herzberg Astronomy and Astrophysics Research Centre's expected results.

Among students, beyond the universities that the Herzberg Astronomy and Astrophysics Research Centre engages with (e.g., University of Victoria, University of Montréal, McGill University and University of Toronto), there is limited awareness of research opportunities at the research centre or options for PDFs.

According to survey respondents, the CADC provides significant support to the astronomy community. However, there is a perception among external stakeholders and the PRC that others within the community are not fully aware of all services that the research centre offers, such as research collaboration opportunities and data support provided by CADC.

A reduction in services and efficiency

IT improvements since the 2016 evaluation include a 10 gigabyte (GB) internet upgrade at the DAO, a 1GB internet upgrade at the DRAO and increased storage capacity for the CADC. Despite improvements, challenges persist.

According to research centre staff, centralization of IT services under SSC has led to a reduction in services and efficiency for the research centre. 

There are few on‑site staff that are equipped to deal with the research centre's specialized dynamic requirements and SSC staff lack scientific expertise to respond to the research centre's specialized technical needs.

Because SSC's data storage is insufficient for the quantity of data held at the CADC, the research centre has partnered with Compute Canada to use a cloud processing system to support data storage needs.

Some NRC services create challenges for the research centre

The NRC's network is slow due to NRC IT security features, which has led to inconsistent connections and video quality during virtual meetings, creating communication challenges when working with external partners.

The research centre has experienced long delays in the procurement of basic workplace technology devices, such as laptops for researchers.

With the centralization of NRC services, the research centre lost dedicated support specialists. Hiring, onboarding, travel arrangements and ergonomic assessments have been delegated to researchers and supervisors, increasing workload and shifting priorities away from research and management to administrative tasks.

The NRC must follow the Government of Canada's standardized formatting for all research centres' webpages. As content is the responsibility of the Communications Branch, the research centre must seek their approval, which can affect their ability to provide rapid updates and specific content.

The PRC found that the research centre should improve its communication of both outcomes and successes (through its website) to external stakeholders.

The research centre provides researchers access to both domestic and international observatories

The Herzberg Astronomy and Astrophysics research centre has a strong track record in managing national astronomy facilities and in participating in international facilities.

According to the NRC facility reviews, the research centre's facilities are unique in Canada, providing researchers with core capabilities and expertise in the development of observatory instrumentation, however some facilities are in need of maintenance or upgrading (e.g., DAO 1.8‑m Plaskett and DRAO Synthesis Telescope).

The PRC reported that the research centre has invested in the infrastructure necessary for future projects such as the new integration facility in Victoria, BC.

Located on the DRAO site and completed in 2017, CHIME, a university owned and Herzberg Astronomy and Astrophysics Research Centre supported telescope experiment, was designed to learn more about dark energy. There are no other comparable radio telescopes to the CHIME in the northern hemisphere.

CADC is critical to research needs, some improvements underway

82% of survey respondents reported that access to the CADC is critical to their research needs. A small number (n=10 or 13%) commented that the user interface and search engine of the CADC could be improved.

Suggestions include improving the ability to select data based on more specific criteria, a multi‑use hub that analyzes and processes data with Python and a more responsive search engine to access and extract data. The research centre's staff report that many of these improvements are underway.

 

Figure 8 – Text version

A vertical bar chart that depicts the proportion of survey respondents that were satisfied with the efficiency of the application process for accessing the research centre's domestic and internal telescopes.

The data show that survey respondents were satisfied with the efficiency of the application process for accessing the research centre's domestic and internal telescopes.

• 93 percent of survey respondents were satisfied with the application process to access the ALMA telescope.
• 93 percent of survey respondents were satisfied with the application process to access the CFHT telescope.
• 89 percent of survey respondents were satisfied with the application process to access the Gemini telescope.
• 100 percent of survey respondents were satisfied with the application process to access the DAO telescope.
• 100 percent of survey respondents were satisfied with the application process to access the DRAO telescope.

 

Canada's participation in international telescopes relies on 2 Gs&Cs programs

The Herzberg Astronomy and Astrophysics Research Centre receives funding for Canada's participation in international telescopes. The 2 programs are:

• Annual International Telescope Grants & Contributions (Gs&Cs) Program: provides an annual allocation to fund Canada's participation in international telescopes: ALMA, Gemini, CFHT and supports Canada's participation in SKA.
• TMT (Gs&Cs) Program: funds Canada's participation in the development of the TMT. The Herzberg Astronomy and Astrophysics Research Centre leads Canada's participation in TMT and the research centre's Director General is a standing Board Member on the TMT International Observatory.

 

Figure 9 – Text version

A vertical bar chart that depicts the grant and contribution values for the international telescopes for 2020‑2021.

• Grant and contributions for TMT^{Table note *} totaled $9,406,145.
• Grant and contributions for Gemini totaled $7,834,804.
• Grant and contributions for CFHT totaled $4,608,441.
• Grant and contributions for ALMA totaled $1,448,987.
• Grant and contributions for SKA totaled $736,981.

Table notes

Table note *

TMT contributions began in 2018‑19

Return to table note * referrer

 

Challenges surrounding TMT and SKA

There have been significant challenges with the construction of TMT (e.g., delayed Mauna Kea access negotiations, rising project costs and partners shifting funding to health related fields) and with the SKA (e.g., construction delays and funding uncertainty).

Although the factors affecting these projects are beyond the control of the research centre, it seeks to mitigate risks by contributing to constructive dialogue with international partners.

• in the case of TMT, the research centre continues design work and supports project‑wide efforts to secure funding
• for SKA, the Herzberg Astronomy and Astrophysics Research Centre is exploring other applications for already developed technology to preserve the value of the investments in instruments designed for SKA

PRC recommends the Herzberg Astronomy and Astrophysics Research Centre use influence with regards to TMT and SKA

The PRC recommended that the research centre continue to exert its influence on these major international projects through its roles on international committees and by working in collaboration with the Canadian community and other related organizations (e.g., CASCA, ACURA) to ensure Canada's future participation and to preserve Canada's significant investment in these projects.

In addition, the research centre should clearly communicate the risks associated with these larger projects to the Canadian astronomy community to ensure that the community's expectations for the research centre's resources are managed.

Overall satisfaction with PDF program, but some challenges

Students were generally satisfied with the co‑op and graduate selection process. The research centre offers very attractive PDF positions with competitive funding.

Students and PDFs reported that the Herzberg Astronomy and Astrophysics Research Centre has had a positive impact on their early careers by providing:

• flexibility to choose their research interests
• access to opportunities in a large variety of research areas
• ability to network and participate in research collaborations
• sufficient funding for travel and conference fees

The selection process for PDFs, however, has some challenges:

• the NRC posts PDF job opportunities according to its internal schedule, which is not aligned with the general astronomy PDF hiring season which occurs in the fall and winter
• this misalignment reduces the candidate pool as many of the PDFs have already been hired elsewhere, by the time the research centre posts their available positions

Increasing awareness and opportunities

The Herzberg Astronomy and Astrophysics Research Centre could increase awareness of opportunities among students, both nationally and internationally, which could attract more women and minority‑group students to the field of astronomy and astrophysics.

There may be opportunities to provide students with greater industry exposure and technical career development.

The PRC found that the research centre could establish a stronger mentoring system for early career researchers, including skill development tools.

Opportunities to increase awareness of services

The research centre offers strong research, technology and data support services and aligns these service with the needs of the Canadian astronomy community. (See section Relevance: Alignment with the Canadian astronomy community)

While the research centre is well known within the Canadian astronomy community, external stakeholders expressed that not all researchers are aware of the full range of services, such as the potential for research collaborations.(See section Relevance: Alignment with the Canadian astronomy community)

The data and technical support provided by the CADC are also seen by many survey respondents (82%) as critical to their research needs, however there is a perception among external stakeholders and the PRC that others within the community are not fully aware of all services that the research centre offers, including the data support provided by CADC.

Astronomy projects are becoming more complex

The research centre's ability to provide greater research support is limited by available resources (i.e., budget and staff). The availability of resources determines which projects are supported and the scope of the research centre's involvement. (See section Capabilities: Project size and selection)

In general, astronomy projects are becoming larger, more complex and more expensive. As a result, the research centre developed a project selection process whereby project participation is assessed against the research centre's strategic plan and available resources. (See section Capabilities: Project size and selection)

The research centre plays a significant role in Canada's position

Canada has maintained its overall high ranking position in astronomical science over the evaluation period. External stakeholders rank Canada highly, as do independent web rankings, ranging from 3rd – 6th, from The World University Rankings, Scimago Journal and Country Ranking, and Kenznow.

83% of survey respondents indicated that the research centre has been an important factor in Canada's performance in astronomy.

According to the PRC, the research centre delivers high impact for the Canadian astronomy community and its expertise is world‑leading and highly regarded (e.g., expertise in high contrast imaging for exoplanet research, its support of high performance computing and legacy data resources, and its active role in both science and technical support and development for CHIME).

Factors affecting Canada's future in astronomy

Although the research centre consistently participates in high‑impact projects, it is challenging for the research centre to commit to these large, long‑term projects, as Canada does not have a clear process to fund them.

While Canada continues to move forward in astronomical science, there is concern among external stakeholders that other countries are moving faster and that due to the present funding structure, Canada is struggling to keep pace.

The fate of the TMT and Canada's funding decisions around SKA are critical in determining Canada's position in the field, which by extension could impact the research centre.

Stakeholders recognize that should TMT remain stalled and Canada not engage financially in SKA, it would be a significant setback in light of all the time and resources the Herzberg Astronomy and Astrophysics Research Centre has invested into these longstanding complex projects.

"The Herzberg Astronomy and Astrophysics Research Centre should preserve and expand its core forward‑looking technology and scientific research that will bolster the long‑term success of the research centre and Canadian astronomy overall."

Challenges

COVID‑19 has affected the research centre in a number of ways, including allowing fewer opportunities for students and PDFs to network or to form critical relationships with project partners.

Travel restrictions have affected international projects such as the installation of new technology developed for GEMINI South.

In addition, the ability to attract international PDFs has been curtailed.

Opportunities

Despite these challenges, the research centre met and in some instances exceeded its targets during COVID‑19 (e.g., publications, number of collaborators working with the research centre).

COVID‑19 also provided new opportunities for staff, including the development of targeted virtual groups such as the Optical Spectroscopy group, as well as greater interactions between DAO and DRAO staff (e.g., DVA2, 26‑m upgrade, ARTTA‑4 and the SKA digitizer).

Strategic Plan

The PRC recommended that both the LRP 2020 and the latest developments on the TMT and SKA be included in the research centre's strategic plan.

Increase awareness

Both internal and external stakeholders reported limited awareness of the research centre within the international astronomy community, industry and the public.

The PRC suggested that the research centre improve communication of its outcomes and successes to external stakeholders and the public. The committee also recommended the research centre develop a targeted strategy for connecting with industry to enhance staff capacity and skills.

There are examples of collaboration between the Herzberg Astronomy and Astrophysics Research Centre and other NRC research centres. There may be opportunities for further collaboration among NRC research centres where areas of research overlap.

Improve IT solutions

Both staff and OGDs indicated that there are ongoing challenges with IT support for the Herzberg Astronomy and Astrophysics Research Centre. Although there have been some IT improvements since the 2016 evaluation, there continues to be a reduction in services and efficiency, which has had a negative impact on the operations of the research centre.

Increase diversity

The Herzberg Astronomy and Astrophysics Research Centre workforce is currently under‑represented in most employment equity targets groups including women in middle and senior management. The PRC also recommended that workforce diversity should be improved when filling positions vacated by retirement and that the research centre aim to attract high‑profile diverse talent for high‑level positions.

Early career mentoring and development

The PRC found that the research centre could be more proactive and impactful for its staff and the Canadian astronomy community in mentoring and career development. This could include working with universities to ensure appropriate mentoring and career development is occurring for Herzberg Astronomy and Astrophysics students. The research centre has the opportunity to become a leader in the professional development of early career researchers in Canada.

Management Response and Action Plan (MRAP)

Document Review

Internal and external documents were reviewed to provide context and to complement other lines of evidence in assessing relevance and performance. Internal documents included the research centre logic model, operational and strategic plans, departmental reports, advisory board terms of reference and presentations, facility reviews and project prioritization documents. External documents included website information, telescope contribution agreements and documents related to government priorities.

Data Review

The research centre's administrative and performance data for 2016-17 to 2020-21 were reviewed to provide information on program inputs (i.e., resources), outputs and client engagement. This included key performance indicators, financial data, human resource data, project data and client data.

Survey of Canadian Astronomers and Astrophysicists

An online survey of Canadian astronomers and astrophysicists was conducted to assess the extent to which the Herzberg Astronomy and Astrophysics Research Centre's activities and objectives met the needs of the Canada astronomy community; the research centre's ability to provide access to national and international telescopes as well as research supports; and the scientific impacts from the telescope access and research supports.

The survey targeted members of ACURA universities or the Centre de recherche en astrophysique du Québec (CRAQ); Canadians researchers that have applied for time on domestic (DRAO, DAO) and/or international telescopes (CFHT, Gemini, ALMA); and researchers who have used CADC data from 2016 to present.

Survey invitations were sent to 767 individuals. A total of 133 respondents completed the survey, representing a 17% response rate.

Internal Focus Groups & Interviews

Internal focus groups and interviews were conducted with a total of 28 individuals to collect information such as personal experiences, opinions and expert knowledge related to the research centre's relevance, engagement, resources, scientific excellence and achievement of expected results.

Focus groups included the research centre's Directors and the Head of the CADC (n=6), the Herzberg Astronomy and Astrophysics Advisory Board (n=6), Researchers and Technicians (n=5), students and PDFs (n=5). Interviews included the Director General of the Herzberg Astronomy and Astrophysics Research Centre, the Vice‑President of Emerging Technologies and other key internal individuals (n=6).

Information was used to complement other lines of evidence and to contextualize quantitative information.

External Interviews

External interviews were conducted with stakeholders to collect qualitative evidence with regards to the research centre's relevance, engagement, facilities, scientific excellence and achievement of expected results.

A total of 16 external interviews were completed, including academics (n=9), industry partners (n=2), other federal government departments or agencies (n=2) and telescope directors (n=3).

Information gathered through the interviews provided contextual information and was used in conjunction with the other lines of evidence.

Peer Review Committee

A peer review committee was convened to assess the research centre's relevance, engagement and scientific excellence. The committee was composed of 6 members with expertise in astronomy and astrophysics (see appendix B for PRC members' biographies).

Members included national and international representatives from academia, research organizations and industry. To ensure objectivity and avoid conflicts of interest, peer review committee members signed a confidentiality and conflict of interest agreement.

Peer review members examined key research centre documents, the preliminary evaluation findings and facility reviews prior to participating in 4 virtual review sessions. Virtual sessions included presentations and poster sessions delivered by the research centre, a student session and discussions with the Herzberg Astronomy and Astrophysics Research Centre's Director General and senior management.

Based on the documents reviewed and the virtual sessions, the peer review committee produced a report that responded to each of the areas under assessment. This report was used as a line of evidence in the overall evaluation findings.

Limitations and mitigation strategies

Although the evaluation encountered some challenges, methodological limitations were mitigated, where possible, through the use of multiple lines of evidence and the triangulation of data. This approach was taken to establish the reliability and validity of the findings and to ensure that conclusions and recommendations were based on objective and documented evidence. Details on limitations and their associated mitigation strategies are described below.

Dr. Carole Jackson (Committee Chair)

Dr. Carole Jackson is the British/Australian past Director General of ASTRON, the Netherlands Institute for Radio Astronomy based at Dwingeloo in the Netherlands. She is an expert in extragalactic radio astronomy and technology research management, gained from working in commerce, industry and the research sectors through her long career. Whilst working for CSIRO Astronomy and Space Science (2003‑2013), Dr. Jackson led the design and delivery of the 36 antennas forming the core part of the Australian SKA pathfinder telescope (ASKAP). She has a long association with the International Square Kilometre Array (SKA) project where she has worked closely with the Australian, and more recently, Dutch, governments to guide their business cases towards investing in the SKA.

Dr. Stefi Baum

Dr. Stefi Baum is currently on sabbatical from the University of Manitoba after serving as Dean of the Faculty of Science and Professor of Physics and Astronomy since October 1, 2014. She came to the University of Manitoba following ten years at the Rochester Institute of Technology (RIT) where she served as Professor and Director of the Chester F. Carlson Center for Imaging Science. Prior to RIT, Dr. Baum served just under 2 years as an American Institute of Physics Science Diplomacy Fellow at the U.S. Department of State where she worked to promote agricultural science and food security in developed and developing countries.

Dr. Mark Bentum

Dr. Mark Bentum received the PhD degree from the University of Twente, Enschede, The Netherlands in 1995. In 1996 he joined the Netherlands Foundation for Research in Astronomy (ASTRON). From 2005 to 2008 Dr. Bentum was responsible for the construction of the first software radio telescope in the world, LOFAR (Low Frequency Array). In 2008 he became an Associate Professor in the Telecommunication Engineering Group at the University of Twente. From December 2013 to September 2017 he was also the program director of Electrical Engineering at the University of Twente. In 2017 Dr. Bentum became a Full Professor in Radio Science at Eindhoven University of Technology. He is now involved with research and education in radio science. His current research interests are radio astronomy, short‑range radio communications, novel receiver technologies (for instance in the field of radio astronomy), channel modeling, interference mitigation, sensor networks and aerospace. Since 2020 he is the head of the Astronomy & Operations department at ASTRON.

Dr. Makenzie Lystrup

As Vice President and General Manager of Civil Space, Dr. Mackenzie Lystrup is responsible for Ball Aerospace's portfolio of civil space systems that include science, operational weather and Earth observation, and advanced technologies development objectives. Dr. Lystrup joined Ball Aerospace in January 2013 in the company's Strategic Operations organization, based in Washington, D.C. She most recently served as Senior Director, Civil Space Advanced Systems and Business Development. Previously, Dr. Lystrup worked in the U.S. House of Representatives as a Congressional Science & Technology Policy Fellow. During her time, she managed an issue portfolio ranging from technology and privacy to national defense to nuclear energy and nonproliferation. In 2019, Dr. Lystrup was elected to the rank of Fellow of the American Association for the Advancement of Science for her distinguished record in the fields of planetary science and infrared astronomy, science policy and advocacy and aerospace leadership.

Dr. Mark McKinnon

Dr. Mark McKinnon is the Assistant Director for New Mexico Operations with the National Radio Astronomy Observatory in Socorro, New Mexico, USA. His department operates and maintains the Very Large Array and Very Long Baseline Array radio telescopes. He has over 25 years of experience in the design, construction, operation, and maintenance of large, ground‑based radio astronomy facilities. Dr. McKinnon has held project management roles in the construction of the Green Bank Telescope, the Expanded Very Large Array, and the Atacama Large Millimeter Array. He has also led the Dish Consortium for the Square Kilometer Array. His research interests include pulsar astrophysics, polarimetry and statistics.

Dr. Suzanne Ramsay

Dr. Suzanne Ramsay is an astronomer working at the European Southern Observatory headquarters outside Munich, Germany. Dr. Ramsay is a specialist in infrared and optical ground based instrumentation and has delivered instruments and systems to the UKIRT, Gemini and VLT telescopes. She is the Instrumentation Manager for ESO's Extremely Large Telescope that is currently in development. In this role, Dr. Ramsay is responsible for the delivery of the first suite of instruments to the 39‑m ground‑based telescope, coordinating with consortia of instrument builders in the ESO member states. Dr. Ramsay also maintains an interest in astrophysical research, specifically in the formation of young stars and their impact on their surroundings.

Astronomy Technology R&D and Services

• Adaptive optics
• Digital and analog electronics, signal processing
• Scientific leadership to observatories
• Manufacturing and machining
• Mechanical engineering
• Millimetre technology
• Optical engineering
• Project management and systems engineering
• Radio signal chain, receivers, correlators and digitizers
• Software and controls
• Canadian Time Allocation Committee (CanTAC)
• Solar radio monitoring
• Large‑scale scientific computing infrastructure
• Specialized astronomy data management: data mining, processing, distribution and transfer of astronomical datasets

Scientific R&D

• Solar system origins
• Planetary systems
• Galactic structure
• Evolution of galaxies
• Evolution of structure
• Time domain
• Magnetic fields
• High‑redshift universe

Operations and Support of Ground-Based Telescopes

• DAO – 1.8‑m and 1.2 m Optical Telescopes
• DRAO – 26 m, Synthesis, Solar, CHIME
• Atacama Large Millimeter / submillimeter Array (ALMA)
• Gemini Observatory
• Canada‑France‑Hawaii Telescope (CFHT)
• Canadian Time Allocation Committee (CanTAC)

Scientific Support to Space‑Based Telescopes

• James Webb Space Telescope (JWST)
• Cosmological Advanced Survey Telescope for Optical and UV Research (CASTOR)
• Ultra Violet Imaging Telescope (UVIT)

Dominion Astrophysical Observatory

• Located in Victoria, British Columbia
• Conducts technology research and development associated with instrumentation, facility design and construction projects of the domestic, international and future telescopes they support
• Offers capabilities in optical imaging, spectroscopy and spectropolarimetry
• Includes the 1.8‑m Plaskett Telescope, 1.2‑m Telescope and the CADC

Canadian Astronomy Data Centre

• Established in 1986, this web‑accessible virtual observatory and archive for data has grown to house some of the world's most important astronomical data collections, including those from CFHT, the twin Gemini telescopes and the James Clerk Maxwell Telescope
• Capabilities include large‑scale scientific computing infrastructure and specialized astronomy data management expertise

Dominion Radio Astrophysical Observatory

• Located in Penticton, British Columbia
• Provides scientific and technical services to support Canada's national and international radio telescopes
• Operates several telescopes on its extensive radio‑quiet site
• Features laboratories and specialized equipment for design and construction of radio frequency instrumentation, from highly sensitive antennas and receiver systems to high‑speed digital signal‑processing hardware and software
• Provides science and service support to Canadian university led and owned telescope projects e.g., Canadian Hydrogen Intensity Mapping Experiment (CHIME)

International facilities

Future facilities