On the Job Training
Abstract
We propose a new framework for building and evaluating machine learning algorithms. We argue that many real-world problems require an agent which must quickly learn to respond to demands, yet can continue to perform and respond to new training throughout its useful life. We give a framework for how such agents can be built, describe several metrics for evaluating them, and show that subtle changes in system construction can significantly affect agent performance.
- Publication:
-
arXiv e-prints
- Pub Date:
- June 2005
- DOI:
- 10.48550/arXiv.cs/0506085
- arXiv:
- arXiv:cs/0506085
- Bibcode:
- 2005cs........6085H
- Keywords:
-
- Machine Learning;
- K.3.2
- E-Print:
- 8 pages, submitted to NIPS 2005