Managing global satellite data: The GHRSST-PP

The GODAE (Global Ocean Data Assimilation Experiment) High Resolution Sea Surface Temperature Pilot Project (GHRSST-PP) is an international effort to produce high quality enhanced Level 2 SST products (known as L2P) from a number of satellite infrared and microwave sources on both polar orbiting and geostationary platforms. Ultimately these data will be merged by the project into a daily 10 km global cloud free product. The large volumes of satellite information produced by the GHRSST-PP as well as their timeliness will require coordination among data providers (for each individual satellite sensor) and users, methods of quality control and archiving, and tools for data discovery and distribution. The JPL PO.DAAC has developed an infrastructure to meet the requirements of this project including its stringent realtime nature (data available within 4 hours of satellite downlink). This infrastructure includes dedicated software and hardware to ingest, monitor and track the data and metadata generated from global L2P providers including coordinating data delivery and "hand shaking," staging the data in a 30 day rolling store, constructing custom subsetted regional diagnostic products, and delivery of data products to external users via subscription and other methods. The PO.DAAC has also constructed a metadata repository whereby metadata for each individual L2P product is ingested into a database that is externally accessible through a web-based search and query front end. This metadata repository essentially functions as the data discovery mechanism for all GHRSST-PP products, both L2P and merged, that may be be accessible from a number of global sources (including from JPL). A separate database has been developed for satellite to in situ SST matchup information important for satellite SST validation and algorithm development for the GHRSST-PP science team. We will describe the results of an "end-to-end" test with a provider of L2P data from MODIS and AVHRR that demonstrated the complete cycle of data production, ingest and acknowledgment, server population, metadata population, data discovery and custom product generation.