Common Data and Metadata Models for Geophysical Data in the Cloud

UNAVCO and IRIS, major repositories for global geodetic and seismic data, are in the process of joining their operations to form a unified facility for supporting the broad spectrum of geophysical observations and science required to help understand and predict the behavior of Earth Systems. This process would be complicated in a static data management environment, but both repositories are also migrating archives and services to the cloud as part of the merger. To simplify and unify archive data management, the organizations are collaborating to create common data and metadata models for observations from a wide variety of instruments and disciplines. For data, the initial focus has been on the xArray data model, already used in the geodetic and magnetotelluric communities, which can be implemented with several disc- and cloud-native approaches (HDF5, netCDF4, and Zarr). For metadata, the SensorML Standard developed by the Open Geospatial Consortium is being explored because 1) SensorML accommodates the large parameter space associated with instrument metadata required to use and trust complex observations and 2) the ability to extend the standard when required. The merger of two large repositories combined with migration to the cloud requires careful identification and on-going testing of a wide variety of assumptions about data management systems. This presentation will focus on lessons learned so far.