AIRS Data Product Size Reduction

AIRS standard data products have been designed for ease of use. However, data volume has emerged as a limiting factor in usability and data access. AIRS standard products are designed with full HDF-EOS 4 compliance for its transportability across architectures and for its self-describing nature. In addition, all scientific quantities are expressed simply as floating-point numbers in physical units in order to avoid cumbersome re-scaling of the data prior to use. However, the trade-off has been large data files that can be difficult to use and access because of their size. We have looked at various compression methods to decrease the size of AIRS standard products while maintaining ease of use. Internal HDF compression provides some reduction in file size while remaining transparent to the user, but does not give substantial reduction when used as the only method. Rounding the floating-point numbers by zeroing the least-significant bits of their mantissas allows AIRS products to be compressed significantly without loss of scientific quality. This paper details the trade-offs in choosing how many bits to keep as well as the mechanics of the approach.