Satellite image inpainting using U-Net with partial convolutions: applications on Landsat 8 land surface temperature image patches

Gaps in satellite images caused by cloud contamination, sensor problems, or incomplete satellite coverage must be filled for analyses or models that rely on complete images (e.g., classification and object detection). In this study, we utilize the Land Surface Temperature (LST) image from U.S. Landsat Analysis Ready Data (ARD) to assess the performance of a new model that repairs incomplete satellite images. Complete LST patches (64x64 pixels) for two identical scenes acquired at different dates (up to 48 days apart) were randomly paired with ARD cloud masks to generate the model inputs. Adopting the U-Net architecture, the model replaced the classical 2D convolution layer with the modified partial convolution layer and the derived merging layers to handle the missing values in the encoder-decoder path. Image inpainting using this model was conducted using the LST image patches in Colorado between 2014 and 2018. Preliminary results after a 10-epoch training showed that the model is capable of capturing the high-level semantics from the inputs and bridging the difference in acquisition dates for filling gaps. The transition between the masked and unmasked regions (including the edge area of the image) in the inpainted images is smooth and reflects realistic spatial patterns.