  Hyperdimensional computing (HDC) is a biologically-inspired framework which
represents symbols with high-dimensional vectors, and uses vector operations to
manipulate them. The ensemble of a particular vector space and a prescribed set
of vector operations (including one addition-like for "bundling" and one
outer-product-like for "binding") form a *vector symbolic architecture* (VSA).
While VSAs have been employed in numerous applications and have been studied
empirically, many theoretical questions about VSAs remain open. We analyze the
*representation capacities* of four common VSAs: MAP-I, MAP-B, and two VSAs
based on sparse binary vectors. "Representation capacity' here refers to bounds
on the dimensions of the VSA vectors required to perform certain symbolic
tasks, such as testing for set membership $i \in S$ and estimating set
intersection sizes $|X \cap Y|$ for two sets of symbols $X$ and $Y$, to a given
degree of accuracy. We also analyze the ability of a novel variant of a
Hopfield network (a simple model of associative memory) to perform some of the
same tasks that are typically asked of VSAs. In addition to providing new
bounds on VSA capacities, our analyses establish and leverage connections
between VSAs, "sketching" (dimensionality reduction) algorithms, and Bloom
filters.
