Learning restricted Boltzmann Machines with few latent variables

Abstract

Restricted Boltzmann Machines are a common family of undirected graphical models with latent variables, described by an undirected bipartite graph, with all observed variables in one layer and all latent variables in the other. In this thesis, we give an improved algorithm for learning the structure of a Restricted Boltzmann Machine, given samples generated according to it. Currently, the best algorithms for this task have time complexity O(n[superscript d), where n is the number of observed variables and d is the maximum degree of a latent variable. Let the observed neighborhood of an observed variable be the set of observed variables that are connected to it in the graphical model corresponding to the marginal distribution of the observed variables, and let s be the maximum number of latent variables connected to the observed neighborhood of an observed variable. Then, we give an algorithm with time complexity [mathematical expression], which represents an improvement when s < log₂(d - 1). Furthermore, we relax some of the requirements of current algorithms on parameters of the model: specifically, on the width and on the minimum absolute-value non-zero potential.