Structural Indexing of Satellite Images for High Volume Data Access and Analysis

Increasing data volumes from observational systems has been both a boon and a burden to the research scientist. Ever increasing demands on systems, networks and analysis tools means a proportionally smaller amount of the data is ever retrieved for analysis and direct observation by the a scientist. This is especially true in the satellite arena where the new high-resolution multispectral instruments give rise to archives that are hundreds of petabytes in size. Combine this with the fact that the most interesting research comes from combining the raw satellite observation with model, ground and other data sets that can themselves comprise similar size and it is indeed a daunting problem to extract value from the archive. In order to address this problem and aid in efficient data discovery and analysis the National Geophysical Data Center (NGDC) has developed the Structural Indexing of Satellite Images (SISI) system a CBIR(Content based image retrieval) system which combined with model and other observational databases gives the researcher a tool for focusing on the relevant bits from a mountain of data. This presentation will present the details of the SISI system as employed at NGDC and discuss the techniques, tools and workflow used to support community use of a research archive that is growing quickly towards 100 petabytes. We will present also innovative use cases for the data and how such a system can greatly enhance research in a distributed archive environment.