Acquiring, processing, and analyzing open broadcast science data with extremely low latency at scale with AWS Groundstation and cloud processing.

Many high value data products made available from agencies such as NASA, NOAA, and USGS are broadcast in the clear directly from the satellite. Traditional access patterns involve downlinking the data to an agency ground segment, processing by a science team, and subsequent product distribution through a program such as the NOAA Big Data Initiative or a NASA Distributed Active Archive Center (DAAC).

This talk demonstrates the ability to acquire, process, and analyze open broadcast science data entirely in the cloud. Leveraging the new AWS Elastic Ground Station service, we demonstrate selecting and scheduling satellite acquisitions for open science data then leveraging the scalable nature of the cloud to execute a full processing stack. This talk demonstrates the ability to acquire L0 data, leveraging open data processing algorithms from agencies like NASA and NOAA, and a highly scalable, cost effective, and secure processing architecture to produce near real-time data. By taking advantage of direct satellite acquisition and cloud scale processing, we demonstrate reduced latency from acquisition to data availability for scenarios like disaster response, tipping and cueing, and location monitoring.

The deployed cloud stack leverages open source components and demonstrates ingest and archival of L0 data, processing to L1 products and above, horizontally scaling data processing algorithms, metadata generation, and search and discovery via APIs and a simple serverless UI.