How a cloud based JupyterHub can scaffold researchers' scientific workflows and teaching activities
Abstract
JupyterHub is open-source software that helps both small teams and large organizations by providing convenient access to Jupyter environments. The typical JupyterHub installation starts Jupyter servers and exposes a user interface like JupyterLab where users can run code and perform interactive computation and data analysis. At this point, JupyterHub and its broad ecosystem of open-source software have evolved beyond doing just that: they are now capable of scaffolding researchers' scientific workflows and teaching activities in several ways that are less well-known. JupyterHub is open-source software that can be deployed on local servers, HPC centers and in the cloud, in a vendor-independent way. In this presentation, we will illustrate how a cloud-based installation of JupyterHub is managed for the real-world needs of a collaborative team of researchers and educators. Our needs include: distribute and collaboratively alter course materials to students during a Research Experience for Undergraduates (REU) bootcamp, working with classical desktop applications, providing easy SSH and SFTP access to the cloud hub for file synchronization, using real-time collaboration capabilities in research and teaching, accessing large-scale cloud-native storage, and allocating on-demand computational capacity to process large-scale data in parallel. Our scientific use cases include the analysis of remote sensing data from the ICESat-2, MODIS, Landsat and MAVEN missions, and the development of new software tools for cryosphere and planetary science research. Our main educational use case is teaching modern programming and reproducible research practices to undergraduate research students. These varied scenarios require a flexible environment where we can adapt as our needs evolve, install new tools, and share with team members both data and software tools on the system. We will illustrate various aspects of the process, emphasizing how others can adapt these patterns to their needs. This work was supported by the NSF Earth Cube Program under awards 1928406, 1928374, and a research grant from Amazon Web Services.
- Publication:
-
AGU Fall Meeting Abstracts
- Pub Date:
- December 2021
- Bibcode:
- 2021AGUFMIN55D..03S