  We report on measurements and modeling of the mode structure of tunable
Fabry-P\'erot optical microcavities with imperfect mirrors. We find that
non-spherical mirror shape and finite mirror size lead to loss, mode
deformation, and shifted resonance frequencies at particular mirror
separations. For small mirror diameters, the useful cavity length is limited to
values significantly below the expected stability range. We explain the
observations by resonant coupling between different transverse modes of the
cavity and mode-dependent diffraction loss. A model based on resonant state
expansion that takes into account the measured mirror profile can reproduce the
measurements and identify the parameter regime where detrimental effects of
mode mixing are avoided.
