  We report a high-finesse bow-tie cavity designed for atomic physics
experiments with Rydberg atom arrays. The cavity has a finesse of $51,000$ and
a waist of $7.1$ $\mu$m at the cesium D2 line ($852$ nm). With these
parameters, the cavity is expected to induce strong coupling between a single
atom and a single photon, corresponding to a cooperativity per traveling mode
of $35$ at the cavity waist. To trap and image atoms, the cavity setup utilizes
two in-vacuum aspheric lenses with numerical aperture ($NA$) of $0.35$ and is
capable of housing $NA=0.5$ microscope objectives. In addition, the large
atom-mirror distance ($\gtrsim1.5$ cm) provides good optical access and
minimizes stray electric fields at the position of the atoms. This cavity setup
can operate in tandem with the Rydberg array platform, creating a fully
connected system for quantum simulation and computation.
