A Theory of Feature Learning
Abstract
Feature Learning aims to extract relevant information contained in data sets in an automated fashion. It is driving force behind the current deep learning trend, a set of methods that have had widespread empirical success. What is lacking is a theoretical understanding of different feature learning schemes. This work provides a theoretical framework for feature learning and then characterizes when features can be learnt in an unsupervised fashion. We also provide means to judge the quality of features via rate-distortion theory and its generalizations.
- Publication:
-
arXiv e-prints
- Pub Date:
- March 2015
- DOI:
- 10.48550/arXiv.1504.00083
- arXiv:
- arXiv:1504.00083
- Bibcode:
- 2015arXiv150400083V
- Keywords:
-
- Statistics - Machine Learning;
- Computer Science - Machine Learning