Measurement and comparison of distributional shift with applications to ecology, economics, and image analysis
Abstract
The concentration of a statistical distribution toward a lower bound is a conceptually simple property that lacks a global descriptive statistic. We term this property distributional shift (DS) and derive its measure as a sum of exponentiated cumulative frequencies. We demonstrate DS as a measure of ecological rarity, economic poverty, and resource scarcity. We then use DS as the basis for a comparative statistic, RDS, that measures the relative direction and magnitude of shift between distributions. Weibull random sampling revealed that RDS is strongly related to other comparative statistics, e.g., earth mover's distance, ranked probability score. Applications to image analysis revealed how the directionality of RDS allows it to detect signals, patterns, and visual shifts that other measures do not. Altogether, DS is an intuitive property that underpins a uniquely useful comparative measure.
- Publication:
-
arXiv e-prints
- Pub Date:
- January 2024
- DOI:
- 10.48550/arXiv.2401.11119
- arXiv:
- arXiv:2401.11119
- Bibcode:
- 2024arXiv240111119L
- Keywords:
-
- Statistics - Methodology
- E-Print:
- 32 pages, 3 tables, 7 figures, 35 references, Supplemental figures included