3D Models of the Martian Surface in Virtual Reality

Recent advances in Virtual Reality (VR) hardware allow for immersive visualization of planetary data. Especially the visualization of topography (Digital Elevation Models (DEMs) and 3D models of surface features) in their natural form offers a more intuitive sense of scale and relation. We created a virtual replica of the Jezero crater, which was selected as the landing site for the Perseverance rover. Video: https://t.ly/J4fZ To achieve the highest possible fidelity, we use imagery from the Context Camera (CTX) [1] and High-Resolution Imaging Science Experiment (HiRISE) [2] instruments onboard NASA's Mars Reconnaissance Orbiter [3]. We create initial Stereo DEMs with the Ames Stereo Pipeline (ASP) [4]. Subsequently, we apply our atmospherically corrected Shape from Shading (SfS) method [5], yielding an effective spatial resolution sub 1m/pixel. The user can switch between CTX and HiRISE DEMs and compare the quality. The camera systems on the Mars 2020 Perseverance rover [6] collected images with multiple perspectives. We apply Structure from Motion (SfM) to acquire 3D models at sub 1cm resolution. Our pipeline utilizes the open-source programs Meshlab [7] and CloudCompare [8], as well as the proprietary software Agisoft Metashape [9]. After aligning the rover images, we generate 3D point clouds and use Poisson surface reconstruction [e.g., 10] to obtain 3D meshes. Finally, we decimate and re-mesh the models and generate texture and normal maps. We visualize both DEMs and 3D models using the real-time development platform Unity®, which handles lighting, rendering, and user inputs, and the SteamVR plugin. To add some color to the DEMs, we use rover images and create texture tiles for the terrain. Finally, the feature 3D models are embedded in the base terrain. By selecting a point on a 2D minimap, the user can teleport to that location or enable smooth movement. The program is available online here: https://zenodo.org/record/7993632 [1] Malin, M.C et al. JGR Planets 2007, 112, E05S04. [2] McEwen et al. JGR Planets 2007, 112, E05S02. [3] Zurek, R.W.; Smrekar, S.E. JGR Planets 2007, 112, E05S01. [4] Beyer, R.A el a., Earth Space Sci. 2018, 5, 537–548. [5] Hess, M.; Tenthoff, M.; Wohlfarth, K.; Wöhler, C. J. Imaging 2022, 8, 158. [6] Maki, J.N., et al. Space Sci Rev 216, 137, 2020. [7] Cignoni, P. et al., ERCIM News, 2008. [8] CloudCompare (vers. 2.6). 2015. www.cloudcompare.org [9] AgiSoft PhotoScan Professional (vers. 1.2.6). 2016. www.agisoft.com [10] M. Kazhdan, et al., In Eurographics symposium on Geometry processing, vol. 7, 2006.