We present a fully programmable distributed system architecture based on data compression technology. We have developed a general approach to compression of diverse data from large scientific projects and this paper addresses the appropriate system and scientific constraints together with the algorithm development and test strategy. Algorithms which incorporate scientific knowledge and consume relatively few system resources are preferred over ad hoc methods. This framework has been implemented for the Cosmic Background Explorer (COBE) spacecraft by retrofitting the existing VAX-based data management system with high-performance compression software permitting random access to the data.
Astronomical Data Analysis Software and Systems V
- Pub Date: