A Distributed Infrastructure for Ensemble Experiments

Users as well as developers of Earth System Models (ESMs) rely on an infrastructure, consisting of high-end computing, data storage and network resources to perform complex and demanding simulations. In the past, mainly local resources and infrastructures were used. However, the increasing requirements on computing capability and capacity as well as on data storage facilities often exceed the possibilities of a single center. Moreover, an ensemble simulation, consisting of many individual runs, requires intensive computational power and produces huge amount of data to be post-processed and archived. The processing phase is often integrated in a complex workflow including also pre and post-processing steps that are performed on different machines, potentially at different sites, and often by different scientists. In Europe, there is the need to deploy and to develop technologies in order to provide climate scientists with virtual proximity to distributed computing resources and data. The distributed infrastructure should take advantage from the distributed climate centers belonging to the ENES community and it will leverage the external services offered within the European HPC ecosystem, e.g. today by DEISA2 and in the near future by PRACE. This work describes the deployment of a grid prototype used for verifying if complex workflows, defined within the ESM climate community, can take advantage from the use of a distributed environment through the adoption of Grid technologies. This will concern running ensembles of multi-member (and eventually multi-model) experiments. The activities carried out in this task also aim at looking for making the interaction with and configuration of Grid environments straightforward and thus at improving the uptake of Grid technology on a larger scale. The ESM Grid Environment prototype allows exploiting the GRB grid service, developed at the University of Salento, and the basic services offered by the Globus Toolkit middleware or directly through SSH/SCP. It has been designed to be a component of the IS-ENES virtual Earth System Modelling Resource Centre (vERC) which aims at integrating the European ESMs and their hardware, software and data environments, compliant also with the DEISA2 and PRACE infrastructures. A workbench framework, based on web technologies, provides the users with the access to the computational power of the infrastructure. The framework provides tools to customize Grid users' applications, to manage Grid resources and to support the development cycle of new Grid applications. With the help of this workbench not only Grid application users but also resource providers and application developers are supported in their interactions with the Grid environment. It has been developed by the CMCC (SCO Division). The prototype has been deployed involving three sites composed of the CMCC, DKRZ and BSC nodes. A case study related to a global coupled ocean-atmosphere general circulation model (AOGCM) developed by the CMCC (ANS Division), has been considered and preliminary tests carried out demonstrated the validity of the proposed solution.