《Efficient Learning of Deep Boltzmann Machines_AISTATS2016》.pdf

Efficient Learning of Deep Boltzmann Machines Ruslan Salakhutdinov Hugo Larochelle Brain and Cognitive Sciences and CSAIL, Department of Computer Science, Massachusetts Institute of Technology University of Toronto rsalakhu@ larocheh@ Abstract Many existing machine learning algorithms use “shallow” architectures, including neural networks with only one hid- den layer, kernel regression, support vector machines, and We present a new approximate inference algo- many others. Theoretical results show that the internal rep- rithm for Deep Boltzmann Machines (DBM’s), resentations learned by such systems are incapable of ef- a generative model with many layers of hid- ficiently extracting some types of complex structure from den variables. The algorithm learns a separate rich sensory input (Bengio LeCun, 2007). Training these “recognition” model that is used to quickly ini- systems also requires large amounts of labeled training tialize, in a single bottom-up pass, the values of data. By contrast, object recognition in the visual cortex the latent variables in all hidden layers. We show