Development of an Online Archive for Terrestrial Lidar Data

The UNAVCO Geodetic Imaging program provides terrestrial laser scanning (TLS) support to the Earth science community through TLS instrumentation, field engineering, data processing, and technical training. As part of this TLS support role, UNAVCO is responsible for generation of level one (L1) TLS data products and TLS data archive and access. A UNAVCO-organized TLS community workshop held October, 2011 in Boulder, Colorado defined many of the challenges and requirements a TLS data archive and access system must address. TLS data acquisition presents unique challenges for metadata, provenance capture, and data archive: datasets are often large (several Gb per day) and are stored in a variety of proprietary formats that require conversion and standardization for access and exchange. Due to the wide range of scientific and engineering objectives that motivate TLS data collection, field methods and collection techniques vary greatly and must be thoroughly documented in project metadata. These challenges make level zero (L0 - raw) data capture, metadata preservation, and data provenance important objectives for the TLS repository. To address this set of challenges and related requirements, UNAVCO is developing a TLS repository based on the open source RAMADDA platform (http://ramadda.org). The UNAVCO TLS repository will provide online archive of L0 and L1 data products, capture field metadata, document data processing workflows for provenance, and store original georeferencing information. In addition, the TLS repository provides on-demand services for georeferencing, simple point cloud visualization, data sub-setting and thinning, and file format (e.g., LAS, ASCII, proprietary) data conversion. The RAMADDA-based system offers automation of RINEX processing of GPS data, OPUS and CSRS submission and solution ingestion, and generation of control point lists to streamline the georeferencing of TLS data. Georeferencing metadata and GPS file provenance are particularly important for scan reoccupations and repeat scanning for geodetic studies. Each RAMADDA service logs its processing workflow and related parameters to retain provenance and allow for constraints on error budgets and transparent reprocessing by future investigators. In addition to storing the L0 raw scan data in proprietary format, the TLS repository will also host UNAVCO's standard L1 data product, a merged, aligned point cloud in an open format (LAS or ASCII). This product is the most accessible and useful starting point for supported investigators conducting TLS-based science. While directly accessible to the UNAVCO community, the RAMADDA system also provides service-level access, enabling an external client such as the OpenTopography lidar data facility to display metadata and directly access data from the UNAVCO-based repository.