Improving application of data quality information in accessing and using satellite data

A recurring demand in working with satellite-based earth science data records is the need to apply data quality information. Such quality information is often contained within the data files as an array of "flags", but can also be represented by more complex quality descriptions such as combinations of bit flags, or even other ancillary variables indicating thresholds to be applied to the geophysical variable of interest. For example, with Level 2 granules from the Group for High Resolution Sea Surface Temperature (GHRSST) project up to 6 independent variables could be used to screen the sea surface temperature measurements on a pixel-by-pixel basis. Quality screening of Level 3 data from the Soil Moisture Active Passive (SMAP) instrument can be become even more complex, involving 161 unique bit states or conditions a user can screen for. The application of quality information is often a laborious process until the user understands the implications of all the flags and bit conditions, and requires iterative approaches using custom software. In addition, most visualization packages do not understand how to apply quality information. The Virtual Quality Screening Service, a 2013 NASA ACCESS project, is addressing these issues and concerns. The project has developed an infrastructure to expose, apply, and extract quality screening information, building off known and proven NASA components for data extraction and subset-by-value, implementations of Map Reduce workflows, data discovery, and exposure to the user of granule-based quality information. Further sharing of results through well defined URLs and visualization capabilities will also be described. The presentation will focus on overall description of the technologies and informatics principals employed by the project, and recent results and infrastructure status. Examples of implementations of the end-to-end web service for quality screening with GHRSST and SMAP granules will be discussed.