Rubin Observatory Partners With Google Cloud On Data Facility

Google Cloud announced that the Vera C. Rubin Observatory has entered into a three-year agreement to host its Interim Data Facility (IDF) on Google Cloud Platform. This collaboration will enable Rubin Observatory to make more accurate astronomy predictions and scientific discoveries, ushering in a new era in large-scale scientific computing astronomy workloads. Until the telescope becomes operational in 2023, the IDF will collect astronomical data, or “pre-survey data,” to train the telescope to recognize images of our solar system. The observatory was founded in Cerro Pachón, Coquimbo Región, Chile, due to its prime geographic location for astronomy observations and discovery.

This agreement marks the first time a cloud-based data facility has been used for an astronomy application of this magnitude. The Rubin IDF will process astronomical data collected by Rubin Observatory and make the data available to hundreds of users in the scientific community in advance of Rubin Observatory’s 10-year Legacy Survey of Space and Time (LSST), a planned 10-year survey of the southern sky. A 500-petabyte set of data products and images, the LSST will be used to address the most critical questions about the universe’s structure, evolution, and the objects within it. The IDF will also allow the Rubin operations team and the Rubin science community to be ready when the telescope turns on in 2023.

“We’re extremely pleased to work with Google Cloud on this project.” said Bob Blum, director of operations for Rubin Observatory. “By using an established and trusted cloud infrastructure, we’ll be able to deliver data to the Rubin community that will have an even greater impact on future discoveries of the universe. This is the one experiment that gathers the data on the solar system, all the stars, the galaxy, and the expansion of the universe itself.”

“The advancements we’re seeing in astronomy point to the growing appetite for data that can only be supported by the cloud’s scale and speed,” said Mike Daniels, vice president, Global Public Sector, Google Cloud. “By collaborating with Google Cloud, Rubin Observatory can build in more flexibility for the rising demand of astronomical data, while taking advantage of low-cost cloud data storage. This means Rubin Observatory can use more funds toward discoveries, instead of IT.”

Because the LSST dataset will be too large to be downloaded and stored locally, astronomers will access the data via the browser-based Rubin Science Platform, which will be deployed on the Google Cloud-hosted IDF. With this level of access, the platform will provide the Rubin Observatory user community with astronomy datasets previews and the ability to experience LSST-like data in a real-time analysis environment. The IDF will also be used to install and train the operations team on the Rubin Science pipelines, which will convert pixel data into a database of stars and galaxies.

The Rubin Observatory will use Google Cloud’s Storage, Kubernetes Engine and Google Cloud Compute as the foundation for the IDF project. Google Workspace will enable productivity and collaboration among its scientists and team members. The Google Cloud IDF is also scalable in response to usage, which will be critical when members of the Rubin Science Community begin accessing the data facility in late 2021.