Building a Database from the SDSS Imaging Data
Abstract
We present our solution for organizing high volumes of astronomical imaging data in a relational database focusing on fast data retrieval and Virtual Observatory-style meta-data representation. Previous work by the authors showed that using relational databases for astronomical data can be very useful not only for reduced catalogs like SkyServer \citep{szalay01} but also for high dimensional binary data like spectra \citep[Spectrum Services,][]{dobos04} where meta-data naturally fits into the relational model while storing the binary blobs in the database yields a significant performance gain over ordinary flat files when batch data processing is an objective. We used a special high-bandwidth data transport protocol for downloading the whole SDSS dataset over the Internet, or about 8 TB of corrected imaging data. All necessary meta-data is stored along with the images. For sky coverage representation we used the Spherical Geometry Library of \citet{budavari07} which supports fast lookup of images based on a reference celestial region. We are also working on a convenient user interface for the database as well as a web service that will support programmatic client access to the imaging data with standard Virtual Observatory protocols.
- Publication:
-
Astronomical Data Analysis Software and Systems XVIII
- Pub Date:
- September 2009
- Bibcode:
- 2009ASPC..411..366D