Simulated Annealing and Stochastic Learning in Optical Neural Nets: AN Optical Boltzmann Machine

This dissertation deals with the study of stochastic learning and neural computation in opto-electronic hardware. It presents the first demonstration of a fully operational optical learning machine. Learning in the machine is stochastic taking place in a self-organized multi-layered opto-electronic neural net with plastic connectivity weights that are formed in a programmable non-volatile spatial light modulator. Operation of the machine is made possible by two developments in our work: (a) Fast annealing by optically induced tremors in the energy landscape of the net. The objective of this scheme is to exploit the parallelism of the optical noise pattern so as to speed up the simulated annealing process. The procedure can be viewed as that of generating controlled gradually decreasing deformations or tremors in the energy landscape of the net that prevents entrapment in a local minimum energy state. Both the random drawing of neurons and the state update of the net are now done in parallel at the same time and without having to compute explicitly the change in the energy of the net and associated Boltzmann factor as required ordinarily in the Metropolis-Kirkpartrik simulated annealing algorithm. This leads to significant acceleration of the annealing process. Electronic control of the random light array intensity enables realizing any annealing profile. The detailed distribution of noise sources is found to be immaterial as long as they are identical and independent. Moreover, the annealing results are insensitive to the cooling profiles. (b) Stochastic learning with binary weights. Learning in opto-electronic neural nets can be simplified greatly if binary weights can be used. It would pave the way to using binary spatial light modulators such as the Magneto-Opto Spatial Light Modulator (MOSLM). The findings show that the binary weight learning scheme can have a learning score as high as 100%. The findings also show that the introduction of noise in G -space, i.e., annealing in G-space, is required for the learning process. A third development, that is the development of schemes for driving and enhancing the frame rate of magneto-optic spatial light modulators, can make the machine learning speed potentially fast. Details of these developments together with the principle, architecture, structure, and performance evaluation of this machine will be given. In the course of the dissertation, architectures and methodologies of employing optic and electronic components to build neural networks which can learn from examples in accordance to the Boltzmann machine learning algorithm are investigated. As a result of this dissertation, some issues of applying them to large size networks are addressed.