Using Machine Learning for processing Big Data of Copernicus Satellite Sensors at the Example of the TROPOMI/Sentinel 5 Precursor (S5P) Cloud Product

The satellites of the Copernicus program show the increasing relevance of properly handling the huge amounts of data, nowadays common in remote sensing. This is further challenging if the processed data has to be provided within near real time requirements (NRT), like the cloud product from TROPOMI / Sentinel 5 Precursor (S5P).

In order to solve the inverse problems that arise in the cloud product of S5P as well as in remote sensing in general, usually complex radiative transfer models (RTMs) are used. These are very accurate, however also computationally very expensive and therefore often not feasible in combination with NRT requirements. With the recent significant breakthroughs in machine learning, easier application through better software and more powerful hardware, the methods of this field have become very interesting as a way to improve the classical remote sensing algorithms.

In this presentation we show how artificial neural networks (ANNs) replace the original RTM in the ROCINN (Retrieval of cloud information using neural networks) algorithm (which is the main part of the operational S5P cloud product) with sufficient accuracy while at the same time increase the performance by several orders of magnitude. The complete procedure, which consists of sampling and scaling of the training data, the selection of the ANN architecture and the training itself, is explained and the impact of each step on the final results is evaluated. As ANNs have no or just an implicit representation of the underlying physics, we also show how this can lead to unexpected effects. In case of ROCINN this becomes an issue for nadir scenes where the results of the RTM are in general independent of the viewing zenith angle. This, however, is not correctly modeled by the ANN which can lead to further problems. Solutions are the adaption of the sampling as well the combination of different ANNs.

With the example of ROCINN, as part of the operational S5P cloud product, we show the great potential of machine learning techniques in improving the performance of classical retrieval algorithms and thus increasing their capability to deal with much larger data quantities. However, we also highlight their limitations, which can be crucial under certain conditions, and ways to cope with them.