The Outer Rim Simulation: A Path to Many-core Supercomputers

We describe the Outer Rim cosmological simulation, one of the largest high-resolution N-body simulations performed to date, aimed at promoting science to be carried out with large-scale structure surveys. The simulation covers a volume of (4.225 Gpc)³ and evolves more than one trillion particles. It was executed on Mira, a BlueGene/Q system at the Argonne Leadership Computing Facility. We discuss some of the computational challenges posed by a system such as Mira, a many-core supercomputer, and how the simulation code, Hardware/Hybrid Accelerated Cosmology Code, has been designed to overcome these challenges. We have carried out a large range of analyses on the simulation data and we report on the results as well as the data products that have been generated. The full data set generated by the simulation totals more than 5 PB of data, making curation and handling of the data a large challenge in itself. The simulation results have been used to generate synthetic catalogs for large-scale structure surveys, including Dark Energy Spectroscopic Instrument and eBOSS, as well as experiments on the cosmic microwave background. A detailed catalog for the data challenges of the Large Synoptic Survey Telescope Dark Energy Science Collaboration has been created as well. We publicly release some of the Outer Rim halo catalogs, downsampled particle information, and lightcone data.