The All-Sky Virtual Observatory AAT Node

We discuss the motivation and development of the Anglo-Australian Telescope node of the All-Sky Virtual Observatory¹. The project aims to provide IVOA-compliant access to all AAT and UKST data including legacy, ongoing and future surveys. The data to be included is in a wide range of formats, ranging from tabular, FITS and HDF5. The primary challenge of the project is therefore to store large amounts of heterogeneous data and then query it transparently. We also plan to develop the capability to cross-match data across different ASVO nodes (SkyMapper, TAO and MWA). Based on a survey of user requirements, we developed a simple data model for our system, which suggests that traditional relational database solutions are not sufficient for our requirements; greater flexibility is required than these systems can offer. We have settled on the Hadoop ecosystem, following an investigation of the various NoSQL options (including MongoDB and Cassandra), as well as SQL systems that allow more flexible schema, such as MariaDB and PostgreSQL. Hadoop gives us the flexibility to include relational data, as well as more document-driven data, and is therefore well matched to our needs.