  The Kitaev model is a beautiful example of frustrated interactions giving
rise to deep and unexpected phenomena. In particular, its classical version has
remarkable properties stemming from exponentially large ground state
degeneracy. Here, we present a study of magnetic clusters with spin-$S$ moments
coupled by Kitaev interactions. We focus on two cluster geometries -- the
Kitaev square and the Kitaev tetrahedron -- that allow us to explicitly
enumerate all classical ground states. In both cases, the classical ground
state space (CGSS) is large and self-intersecting, with non-manifold character.
The Kitaev square has a CGSS of four intersecting circles that can be embedded
in four dimensions. The tetrahedron CGSS consists of eight spheres embedded in
six dimensions. In the semi-classical large-$S$ limit, we argue for effective
low energy descriptions in terms of a single particle moving on these
non-manifold spaces. Remarkably, at low energies, the particle is tied down in
bound states formed around singularities at self-intersection points. In the
language of spins, the low energy physics is determined by a distinct set of
states that lies well below other eigenstates. These correspond to `Cartesian'
states, a special class of classical ground states that are constructed from
dimer covers of the underlying lattice. They completely determine the low
energy physics despite being a small subset of the classical ground state
space. This provides an example of order by singularity, where state selection
becomes stronger upon approaching the classical limit.
