Bridging Legacy Scientific Software and Cloud-Native Data with Open-Source Application Streaming

The migration to cloud-based data storage, away from local data storage for scientific data, has seen the rapid adoption of cloud-ready data analysis tools such as Jupyter Notebook, and cloud-ready libraries for data reading/writing, such as NetCDF/ncZarr. While these tools and libraries provide cloud-native approaches to working with large amounts of data stored in the cloud, they do not necessarily help with the large number of existing desktop/legacy applications which are currently in use. Even though it is possible for scientists to adapt their approach using these new tools, the resources required for this may not be readily available. In these cases, a stop-gap approach which allows desktop software to run in the cloud may be necessary. Such an approach will allow existing scientific tools to continue in use, while new cloud-native approaches are being developed.

Application Streaming is a familiar approach to running software in remote environments, under circumstances such as those described above. It can be difficult to create an application streaming or virtual environment on an as-need-be, bespoke basis. It can also be an inefficient use of time that might be better spent addressing the science at hand. There are a number of commercial software packages and service providers which provide products which ease this burden, but they potentially carry a burdensome cost. It would be beneficial if there were an open-source, turn-key solution.

With Cloudstream, an open-source, Docker-based Application Streaming framework developed at Unidata, it is possible to use existing data analysis and visualization tools in a cloud-based environment. This work will discuss the underlying technologies used by Cloudstream, and outline how to use Cloudstream to run and access an existing software and tools hosted in commercial, private, or hybrid clouds.