Archival of Non-standard Data Imagery at NOAA's NCEI

The goal of data archival is to preserve data and associated information for future users, so that they can fully make use of the data independently from the original data producer. To fulfill this goal, the NOAA National Centers for Environmental Information (NCEI) diligently work to ensure well-documented data is archived in National Archive and Records Administration (NARA) recommended formats.

The growth of technology has resulted in the production of proprietary data that fall outside the definition of NARA-recommended formats. However, it may not be desirable in all cases to convert the data to an approved archival format, as the conversion may result in the data being difficult to work with, or disseminated effectively.

An example of the archival of data preserved in a "non-archival" format is the collection of coral histology slide imagery in .SVS format, a semi-proprietary tiled image format. Many species of corals are now listed as either threatened or endangered. This classification is critical to help preserve these valuable ecosystems. However, it creates challenges for scientists collaborating on the study of coral diseases, because of restrictions about the transport of physical specimens under the Convention on International Trade in Endangered Species (CITES). Therefore, digital images of coral histology slides have become an invaluable mechanism for scientists to collaborate on coral health issues. Preservation and access to these images is important to allow the larger community to understand how coral disease has changed over both time and location.

The .SVS format is based on the .TIFF file format. Each .SVS file contains multiple images and indexing information. Currently, conversion of these files to an archival format could result in files larger than the originals, and loss of utility of the files. Also, the original metadata is linked to each specific .SVS file.

Working with the data provider, NCEI's data and technology stewards were able to develop a path to preserve this data collection. A key factor of the project was the planning, and decision to include the source code for software needed to read the images. Inclusion of the code increases the probability that the images will be readable in the future, even if specialized viewers were no longer available.