Heuristic-Based Weak Learning for Automated Decision-Making
Abstract
Machine learning systems impact many stakeholders and groups of users, often disparately. Prior studies have reconciled conflicting user preferences by aggregating a high volume of manually labeled pairwise comparisons, but this technique may be costly or impractical. How can we lower the barrier to participation in algorithm design? Instead of creating a simplified labeling task for a crowd, we suggest collecting ranked decision-making heuristics from a focused sample of affected users. With empirical data from two use cases, we show that our weak learning approach, which requires little to no manual labeling, agrees with participants' pairwise choices nearly as often as fully supervised approaches.
- Publication:
-
arXiv e-prints
- Pub Date:
- May 2020
- DOI:
- 10.48550/arXiv.2005.02342
- arXiv:
- arXiv:2005.02342
- Bibcode:
- 2020arXiv200502342S
- Keywords:
-
- Computer Science - Computers and Society;
- Computer Science - Artificial Intelligence;
- Computer Science - Machine Learning;
- K.4.1;
- I.2.6
- E-Print:
- 5 pages, 3 figures. Camera-ready version for Participatory Approaches to Machine Learning @ ICML 2020. Last updated Dec. 2020: fixed bug in Figure 3 - "always intervene" heuristic should be "never intervene."