Quantifying the Reliability and Robustness of Random Forest and Boosted Decision Tree Models with Large Training Data Sets
Abstract
Machine learning models have assisted us in making discoveries and added overall efficiency in many aspects of life. As we become more reliant on these computer-derived inferences we are compelled to ask whether the reliability and robustness of machine learning models is fully understood. Recent research has focused on measuring the robustness of state-of-the-art models trained on popular benchmark datasets which are curated and developed to minimize the chances of training data being mislabeled or containing anomalous values in its features. Motivated by the fact that real world datasets may contain larger amounts of inaccuracies in labels and naturally occurring perturbations in the training data, we measure three metrics to determine the reliability and robustness in an image classification setting known to have such faults. We measure how well separated are the classes of a training dataset that is composed of randomly sampled MODIS surface reflectance pixels from a geographically diverse set of locations that is aimed to represent a global data distribution. We then benchmark a set of non-parametric learning algorithms such as random forests and boosted decision trees to see how reproducible the image classification task between each algorithm is. Finally, we determine how reproducible these models are when performing fractional random sampling of the total data to produce the training dataset.
- Publication:
-
AGU Fall Meeting Abstracts
- Pub Date:
- December 2022
- Bibcode:
- 2022AGUFMIN33A..03S