Oceanographic data from Rolling Deck to Repository (R2R), 10 years of platform data management and how it relates to airborne data archiving
Abstract
The mission of the Rolling Deck to Repository Program (R2R) is to support acquisition, documentation, preservation, and enhanced usability of underway environmental sensor data from scientific cruises by the U.S. academic research fleet. Although shipboard data and airborne data differ, the processes by which they are collected and processed are similar: both are collected from multiple sensors on a common platform taken along a trackline through space and time. Through years of experience, the Rolling Deck to Repository has created a refined workflow with processes that may be practical to apply to collected airborne data.
We will present some common issues and solutions devised within the program, including managing the inventory, fileset organization, and tracking device histories. For example, R2R runs a full data inventory of each file collected on a cruise. This allows us to work within the database and organize files mapped to a device, files defined as documentation, calibration information, or versions of processed data. Each cruise has an original data distribution (distro) that holds all files collected within the event. This is stored and can be managed by the database records. Once files are identified as part of a device set they are staged for downstream processing or access. Managed correctly, this can greatly add to file organization and consume less system resources. An oceanic cruise or airborne campaign is a set of files captured while on a platform over a specific time range. We conveniently organize data within these bounds. We are starting to see where this concept is less convenient as ships are beginning to send more data in near real time and we want to see the performance of a device over time. We are looking at decoupling the cruise/data relationship as a way to manage data with more flexibly.- Publication:
-
AGU Fall Meeting Abstracts
- Pub Date:
- December 2019
- Bibcode:
- 2019AGUFMIN43C..03C
- Keywords:
-
- 0394 Instruments and techniques;
- ATMOSPHERIC COMPOSITION AND STRUCTURE;
- 1910 Data assimilation;
- integration and fusion;
- INFORMATICS;
- 1926 Geospatial;
- INFORMATICS;
- 4294 Instruments and techniques;
- OCEANOGRAPHY: GENERAL