Characterization of plant diversity by semantic segmentation of hyperspectral imagery using HSI-CNN

Identifying and characterizing plant species is important in plant diversity assessments. Traditional methods often use manual measurements which are time-consuming and necessarily of limited spatial scale. To address this challenge, we utilized a 3D-2D hybrid Convolutional Neural Net to automatically segment plant species in images taken on field plots and used these segmentations to estimate plant cover. Hyperspectral imagery was captured from field plots using a DJI Matrice 600 Pro equipped with a Headwall Nano-Hyperspec [400-1000 nm]. The plots had monoculture and mixed species of legume (balansa clover and hairy vetch), annual grass (triticale), and forb (canola) functional groups. We trained the model using images from monoculture plots and the ground truth masks were created using a color-threshold method that separates canopy cover from bare ground. The model achieved accuracies > 93% in training and test sets for monoculture plots. The per class accuracies for each species of cover crop were 0.90, 0.94, 0.98, 0.97, and 0.98 for soil, balansa clover, canola, triticale, and hairy vetch respectively. A synthetic dataset of multispecies imagery was produced to train the model in an effort to improve model performance estimating plant cover in highly diverse plots. Through analysis of human estimated plant cover vs model predicted plant cover, the model was able to explain >75% variability in predicting plant cover estimates across quadrants of high plant species diversity. Model performance may be improved by addressing the presence of mixed pixels in the hyperspectral imagery or producing ground truth labeled data of multispecies images.