Remote Sensing in the Deep Ocean: Improving Ocean Exploration Video Data Accessibility and Use with Machine Learning Technologies

VVideo labels or annotations constitute subject metadata for video. Annotations are required to find, access and re-use video data. The creations of annotations, if done manually as the current best practice, are labor intensive. For every hour of video, it takes four to eight hours to annotate the video. The QA/QC of these annotations, in addition, can take as much time. The NOAA National Centers for Environmental Information (NCEI) is the NOAA repository for all NOAA data including video data. NCEI have collaborated with NOAA Ocean Exploration to develop mechanisms to improve the video annotations. The NOAA Office of Ocean Exploration and Research collects video data aboard the NOAA Ship Okeanos Explorer utilizing the remotely operated vehicles, Deep Discoverer and Seirios. NCEI hosts the OER video portal which holds over 270TB of data covering over 11,000 hours of deep sea explorations. Subject metadata is harvested from various manual annotation tools including SeaTube (developed by ONC), dive logs, dive summaries, and other pertinent sources. NOAA has partnered with academia and other non-government partners to improve this process including preparing existing imagery data to contribute to machine learning training datasets. Machine learning techniques have the potential to (I) improve the manual annotation process by directing scarce expert time to observations that need annotations (II) quality check annotation contributions (III) automatically assign annotations to the observation after algorithms are developed from training datasets (IV) methodology for reproducible results. This presentation will address some of the recent projects focusing on this work including leveraging cloud resources for video annotations, developing an undergraduate level curriculum to train students to conduct video annotations and aid the development of the training datasets, and leverage computer vision to discern observations that need annotations.