Repository logo
Log in(current)
Repository logoMIT Open ScholarshipDSpace@MIT
  1. Home
  2. MIT Open Access Articles
  3. MIT Open Access Articles
  4. Learning restricted Boltzmann machines via influence maximization

Learning restricted Boltzmann machines via influence maximization

Thumbnail Image
Download
Name

1805.10262.pdf

Description
Submitted version
Size

535.53 KB

Format

Adobe PDF

Checksum (MD5)

299af687550cd49ba49069fe1a0f5bf4

sword-2019-11-15T18:24:08.original.xml (130 B)
Original SWORD entry document
Author(s)
Bresler, Guy
•
Koehler, Frederic
•
Moitra, Ankur
Date Issued
June 23, 2019
Publisher
ACM
Citation
Bresler, Guy, Koehler, Frederic and Moitra, Ankur. 2019. "Learning restricted Boltzmann machines via influence maximization."
Version
Original manuscript
Abstract
© 2019 Copyright held by the owner/author(s). Publication rights licensed to ACM. Graphical models are a rich language for describing high-dimensional distributions in terms of their dependence structure. While there are algorithms with provable guarantees for learning undirected graphical models in a variety of settings, there has been much less progress in the important scenario when there are latent variables. Here we study Restricted Boltzmann Machines (or RBMs), which are a popular model with wide-ranging applications in dimensionality reduction, collaborative filtering, topic modeling, feature extraction and deep learning. The main message of our paper is a strong dichotomy in the feasibility of learning RBMs, depending on the nature of the interactions between variables: ferromagnetic models can be learned efficiently, while general models cannot. In particular, we give a simple greedy algorithm based on influence maximization to learn ferromagnetic RBMs with bounded degree. In fact, we learn a description of the distribution on the observed variables as a Markov Random Field. Our analysis is based on tools from mathematical physics that were developed to show the concavity of magnetization. Our algorithm extends straighforwardly to general ferromagnetic Ising models with latent variables. Conversely, we show that even for a contant number of latent variables with constant degree, without ferromagneticity the problem is as hard as sparse parity with noise. This hardness result is based on a sharp and surprising characterization of the representational power of bounded degree RBMs: the distribution on their observed variables can simulate any bounded order MRF. This result is of independent interest since RBMs are the building blocks of deep belief networks.
MIT Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Massachusetts Institute of Technology. Department of Mathematics
Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory
Massachusetts Institute of Technology. Institute for Data, Systems, and Society
Terms of Use
Creative Commons Attribution-Noncommercial-Share Alike
http://creativecommons.org/licenses/by-nc-sa/4.0/
Persistent DSpace Link
https://hdl.handle.net/1721.1/138053
DOI of Published Version
https://doi.org/10.1145/3313276.3316372
Repository logo
PrivacyPermissionsAccessibilityContact us
Repository logo
Notify us about copyright concerns.