LDMNet: Low Dimensional Manifold Regularized Neural Networks
Abstract
Deep neural networks have proved very successful on archetypal tasks for which large training sets are available, but when the training data are scarce, their performance suffers from overfitting. Many existing methods of reducing overfitting are dataindependent, and their efficacy is often limited when the training set is very small. Datadependent regularizations are mostly motivated by the observation that data of interest lie close to a manifold, which is typically hard to parametrize explicitly and often requires human input of tangent vectors. These methods typically only focus on the geometry of the input data, and do not necessarily encourage the networks to produce geometrically meaningful features. To resolve this, we propose a new framework, the LowDimensionalManifoldregularized neural Network (LDMNet), which incorporates a feature regularization method that focuses on the geometry of both the input data and the output features. In LDMNet, we regularize the network by encouraging the combination of the input data and the output features to sample a collection of low dimensional manifolds, which are searched efficiently without explicit parametrization. To achieve this, we directly use the manifold dimension as a regularization term in a variational functional. The resulting EulerLagrange equation is a LaplaceBeltrami equation over a point cloud, which is solved by the point integral method without increasing the computational complexity. We demonstrate two benefits of LDMNet in the experiments. First, we show that LDMNet significantly outperforms widelyused network regularizers such as weight decay and DropOut. Second, we show that LDMNet can be designed to extract common features of an object imaged via different modalities, which proves to be very useful in realworld applications such as crossspectral face recognition.
 Publication:

arXiv eprints
 Pub Date:
 November 2017
 arXiv:
 arXiv:1711.06246
 Bibcode:
 2017arXiv171106246Z
 Keywords:

 Computer Science  Computer Vision and Pattern Recognition