Hylatis, a Cloud-Based Hyperspectral Image Analysis Toolkit

Hyperspectral imagers collect data in hundreds or thousands of spectral bands at each pixel of a spatial image, resulting in a three-dimensional cube of dense information content. Their massive volumes are a barrier to their use.

Scientists want to be able to analyze these datasets in an efficient, responsive manner and to use terms and operations that make sense in their domain of spectral imagery analysis. E.g., scientists often wish to:

- ss="p1">focus on regions of interest bounded by time, geolocation, wavelength, or other dimensions;

- ss="p1">compare data co-located in time or location, including comparison of hyperspectral and multispectral imagery;

- ss="p1">fuse spectral datasets in arbitrary ways;

- ss="p1">apply statistical and machine learning algorithms to datasets in order to find patterns and correlations;

- ss="p1">and visualize results. Hylatis is a NASA AIST 2016 project to create a toolset for working with hyperspectral and multispectral imagery staged in the cloud. Big data engines are leveraged to provide a truly interactive experience. Hylatis provides a uniform interface to various spectral datasets. Common operations are provided and project specific operations are supported. Another goal is avoidance of data movement and duplication, so that multiple users can read these datasets simultaneously, taking the code to the data. Moreover, Hylatis is a research project about infrastructure for data access and analysis. The Hylatis data model is a domain neutral, mathematical representation of a dataset. A dataset is simply a function from a set of independent (domain) variables to a set of dependent (range) variables, such as a time series of temperatures, where the independent variable is time and the dependent variable is temperature. Hylatis operations on data adhere to mathematical principles and notions, including: selection, projection, join, groupBy, pivot, transpose, currying, bijective functions, and function composition. From these, we create a higher level domain specific language (DSL) for spectral imagery analysis, allowing arbitrary transformations on a dataset to be coded and performed in a mathematical way. While here the application domain is spectral imagery, any domain can write to this platform. This presentation will discuss the Hylatis project: current state, milestones, timeline, and lessons learned.