Improving oceanographic data delivery through pipeline processing in a Commercial Cloud Services environment: the Australian Integrated Marine Observing System
Abstract
The Integrated Marine Observing System (IMOS) is a national project funded by the Australian government established to deliver ocean observations to the marine and climate science community. Now in its 10th year its mission is to undertake systematic and sustained observations and to turn them into data, products and analyses that can be freely used and reused for broad societal benefits. As IMOS has matured as an observing system expectation on the system's availability and reliability has also increased and IMOS is now seen as delivering 'operational' information. In responding to this expectation, IMOS has relocated its services to the commercial cloud service Amazon Web Services. This has enabled IMOS to improve the system architecture, utilizing more advanced features like object storage (S3 - Simple Storage Service) and autoscaling features, and introducing new checking procedures in a pipeline approach. This has improved data availability and resilience while protecting against human errors in data handling and providing a more efficient ingestion process.
- Publication:
-
EGU General Assembly Conference Abstracts
- Pub Date:
- April 2017
- Bibcode:
- 2017EGUGA..19.5833B