Merging the Data Models of NetCDF and DAP: Design Choices and Benefits

Beginning in 2008 OPeNDAP and Unidata have been working on an ambitious project to merge the functionality of two different implementations of the netCDF API into a single body of code. Unidata's implementation reads and writes to disk files while OPeNDAP's reads from data servers that support its Data Access Protocol (DAP). The reasons for combining the two are principally to reduce maintenance costs and delays for the introduction of new features, but a side affect has been to focus both groups on the issues of data model flexibility and simplicity. The netCDF format/API have been used in a wide range of contexts spanning the gamut of earth science disciplines including meteorology, oceanography, et c., as well as GIS applications. The Data Access Protocol has seen a similar breadth of use. Both of these software systems employ general data structuring technology based on well-understood information science principals such as data typing and grouping. However, the actual data models of DAP 2.0 and netCDF 3 are different in some significant ways. In merging the two both will require significant changes. We will discuss the on-going process of deciding which changes should be made, where they should be made and how to implement them without breaking software that uses the existing software and data models. In addition we will discuss the exciting prospects that combining these two libraries will provide, particularly how the combination of hierarchical, relational and array data types can facilitate data fusion.