Collaborative Design of an Image Annotation Tool for Oceanographic Imaging Systems

We present a design for a web-based image annotation interface developed to assist in supervised classification of organisms and substrate for habitat assessment from multiple, heterogeneous oceanographic imaging systems. The interface enables human image annotators to count, identify, and measure targets and classify substrate in a variety of kinds of imagery including benthic surveys and imaging flow cytometry. These annotations are then used to build training sets for supervised classification algorithms for purposes of characterizing community structure and habitat assessment. The Ocean Imaging Informatics team at WHOI used the Tetherless World Constellation's collaborative design methodology to develop shared formal information model and system design that applies to a variety of image annotation use cases. Because the information model represents consensus between researchers with differing instrumentation and science needs, it assists with rapid prototyping and establishes a baseline against which existing and forthcoming image annotation tools can be evaluated. A technology review suggested that there are few general-purpose image annotation tools suitable for annotation of high-volume oceanographic imagery. Most tools require too many steps for operations that must be repeated thousands of times, and/or lack critical features such as display of instrument metadata, QA/QC, and management of annotator tasks. While some of these problems are user interface limitations, others suggest that existing tools are missing critically important concepts. For example, QA/QC appears in our information model as an "activity stream" associated with each image annotation, consisting of events indicating review status, specific image quality issues, etc. The model also includes "identification modes" that contextualize annotations according to the annotator's assigned task, assisting both with interpreting annotations and with providing contextual user interface shortcuts. Focusing on interoperability and web-accessibility means the tool can be used to annotate any collection of web-accessible images, opening up possibilities for cross-institutional collaboration and citizen science. A prototype implementation is already in use for scallop and groundfish surveys and is being extended to support phytoplankton imagery.; High-level Information Model of General-Purpose Image Annotation Tool