Deep Exploration via Bootstrapped DQN
Abstract
Efficient exploration in complex environments remains a major challenge for reinforcement learning. We propose bootstrapped DQN, a simple algorithm that explores in a computationally and statistically efficient manner through use of randomized value functions. Unlike dithering strategies such as epsilon-greedy exploration, bootstrapped DQN carries out temporally-extended (or deep) exploration; this can lead to exponentially faster learning. We demonstrate these benefits in complex stochastic MDPs and in the large-scale Arcade Learning Environment. Bootstrapped DQN substantially improves learning times and performance across most Atari games.
- Publication:
-
arXiv e-prints
- Pub Date:
- February 2016
- DOI:
- 10.48550/arXiv.1602.04621
- arXiv:
- arXiv:1602.04621
- Bibcode:
- 2016arXiv160204621O
- Keywords:
-
- Computer Science - Machine Learning;
- Computer Science - Artificial Intelligence;
- Computer Science - Systems and Control;
- Statistics - Machine Learning