  In this work, we propose an infinite restricted Boltzmann machine~(RBM),
whose maximum likelihood estimation~(MLE) corresponds to a constrained convex
optimization. We consider the Frank-Wolfe algorithm to solve the program, which
provides a sparse solution that can be interpreted as inserting a hidden unit
at each iteration, so that the optimization process takes the form of a
sequence of finite models of increasing complexity. As a side benefit, this can
be used to easily and efficiently identify an appropriate number of hidden
units during the optimization. The resulting model can also be used as an
initialization for typical state-of-the-art RBM training algorithms such as
contrastive divergence, leading to models with consistently higher test
likelihood than random initialization.
