Analysis of On-Line Learning when a Moving Teacher Goes around a True Teacher
Abstract
In the framework of on-line learning, a learning machine might move around a teacher due to the differences in structures or output functions between the teacher and the learning machine or due to noises. The generalization performance of a new student supervised by a moving machine has been analyzed. A model composed of a fixed true teacher, a moving teacher and a student that are all linear perceptrons with noises has been treated analytically using statistical mechanics. It has been proven that the generalization errors of a student can be smaller than that of a moving teacher, even if the student only uses examples from the moving teacher.
- Publication:
-
Journal of the Physical Society of Japan
- Pub Date:
- February 2006
- DOI:
- 10.1143/JPSJ.75.024003
- arXiv:
- arXiv:physics/0509050
- Bibcode:
- 2006JPSJ...75b4003M
- Keywords:
-
- Physics - Physics and Society;
- Condensed Matter - Disordered Systems and Neural Networks
- E-Print:
- 13 pages, 8 figures