  We explore several extensions of the generalized entropy construction of
Lewkowycz and Maldacena, including a formulation that does not rely on
preserving replica symmetry in the bulk. We show that an appropriately general
ansatz for the analytically continued replica metric gives us the flexibility
needed to solve the gravitational field equations beyond general relativity. As
an application of this observation we study Einstein-Gauss-Bonnet gravity with
a small Gauss-Bonnet coupling and derive the condition that the holographic
entanglement entropy must be evaluated on a surface which extremizes the
Jacobson-Myers entropy. We find that in both general relativity and
Einstein-Gauss-Bonnet gravity replica symmetry breaking terms are permitted by
the field equations, suggesting that they do not generically vanish.
