  A key objective in multi-view learning is to model the information common to
multiple parallel views of a class of objects/events to improve downstream
learning tasks. In this context, two open research questions remain: How can we
model hundreds of views per event? Can we learn robust multi-view embeddings
without any knowledge of how these views are acquired? We present a neural
method based on multi-view correlation to capture the information shared across
a large number of views by subsampling them in a view-agnostic manner during
training. To provide an upper bound on the number of views to subsample for a
given embedding dimension, we analyze the error of the bootstrapped multi-view
correlation objective using matrix concentration theory. Our experiments on
spoken word recognition, 3D object classification and pose-invariant face
recognition demonstrate the robustness of view bootstrapping to model a large
number of views. Results underscore the applicability of our method for a
view-agnostic learning setting.
