The Remote NetCDF Invocation (RNI) middleware platform. Making Scientific Datasets Available for Ubiquitous Computing.

Large holding of NetCDF data, such as in the Earth System Grid (ESG) or the Community Spectro-Polarimetric Analysis Center (CSAC) are vast repositories of data, making it if not impossible, but impractical for users to download and replicate the complete database. Furthermore, each individual dataset is a combination of hundreds of individual NetCDF files. Therefore requesting such dataset for analysis is an expensive transaction for individuals seeking ubiquitous computing. Since the current state of networks can provide for access to individual pieces of the dataset with enough reliability and speed, we seek a solution that will avoid the bulk download of the dataset required a priori, and will instead request needed portions of the dataset just-in-time. In order to achieve this, we modify the NetCDF C library to execute Remote NetCDF Invocation (RNI), that is, to operate on remote dataset, over HTTPS and gsiFTP protocols, individual NetCDF Application Programming Interface (API) calls as if they were local. This mechanism resembles the well known Remote Procedure Call (RPC) yet it radically differs on the binding between local and remote operations. Our design is based on the extensibility mechanism provided by the popular OPeNDAP Back-End Server (BES) middleware platform with Globus GridFTP and Apache modules acting as the proxy transport mechanism (binding) between the local and remote transactions. This paper describes the architecture as well as how we address the technical challenges for the complete system.