Demonstrating Condensed Massive Satellite Datasets for Rapid Data Exploration: The MODIS Land Surface Temperatures of Antarctica

Datasets from long-duration satellite programs, such as MODIS or AVHRR, grow continuously, eventually reaching massive sizes. Time-series scientific analyses may be straight-forward when limited to small spatial extents, but data exploration at continental or global scales becomes difficult and time consuming at high spatiotemporal resolutions. Here we present a method for "condensing" massive datasets with the objective of enabling rapid time-series data exploration at high resolutions. The process essentially removes most redundant or unneeded data, leaving only anomalous data of high interest as well as a baseline dataset for comparison. The resulting subsampled dataset can be accessed and analyzed directly via a conventional database system. Our techniques and results are demonstrated using the entire MODIS Land Surface Temperature (LST) data record for Antarctica; 16 years of daily 1 km data is ingested into the system and explored. Algorithmic details, performance metrics, and a variety of possible scientific uses are presented.