Embed to Control: A Locally Linear Latent Dynamics Model for Control from Raw Images
Abstract
We introduce Embed to Control (E2C), a method for model learning and control of non-linear dynamical systems from raw pixel images. E2C consists of a deep generative model, belonging to the family of variational autoencoders, that learns to generate image trajectories from a latent space in which the dynamics is constrained to be locally linear. Our model is derived directly from an optimal control formulation in latent space, supports long-term prediction of image sequences and exhibits strong performance on a variety of complex control problems.
- Publication:
-
arXiv e-prints
- Pub Date:
- June 2015
- DOI:
- 10.48550/arXiv.1506.07365
- arXiv:
- arXiv:1506.07365
- Bibcode:
- 2015arXiv150607365W
- Keywords:
-
- Computer Science - Machine Learning;
- Computer Science - Computer Vision and Pattern Recognition;
- Statistics - Machine Learning
- E-Print:
- Final NIPS version