Extending Ozone Data Processing to the Community

The Ozone Monitoring Instrument Science Investigator-led Processing System (OMI SIPS) has been the central data processing system for OMI since its launch on NASA's Aura spacecraft in July, 2005. As part of NASA's evolution from mission based processing to measurement based processing, we are evolving the system into a Community Oriented Measurement-based Processing System (ComPS). This involves changing focus from the mission (OMI) to the measurement (total column ozone), and a widening of our focus from the mission science teams to the overall scientific community. The current system takes software developed by scientists, dispatches and executes the software on a compute cluster, archives the results and distributes them to numerous parties. Although this works well for the production environment, access to centralized system has been naturally limited. Ideally, scientists should be able to easily get the data, run their software, make changes and repeat the process until they are happy with the solution to the problems they are trying to solve. In addition it should be simple to migrate research improvements from the community back into the formal production system. Through NASA's "Advancing Collaborative Connections for Earth-Sun System Science," we have extended publicly accessible interfaces into the production system. The system provides an open API via a set of SOAP/XML and REST based web services, enabling scientists, researchers and operators to interact directly with the data and services offered by the central system. The system includes metadata, archive, and planner subsystems. The metadata server stores metadata and provides the ability for processing software to evaluate production rules to determine the appropriate input data files for a given data processing job. The archive server stores the data files themselves and makes it easy for clients to retrieve the files as needed. The planner plans out the set of jobs to be run in the production system and tracks the work being planned and executed. A local version of the planner is part of client software residing on the remote scientist's workstation. It can communicate with the metadata and archive servers to evaluate production rules and retrieve needed data from the central system. It can execute the production versions of certain algorithms, or locally developed or enhanced research versions. Since the algorithms are identically integrated, they can be very easily migrated into the production environment for large scale bulk processing when appropriate. This system provides the scientific community with flexible, data processing capabilities that are scalable for development, research and production environments. This will accelerate the use and validation of the data for science and applications.