  Maximum likelihood estimation in logistic regression with mixed effects is
known to often result in estimates on the boundary of the parameter space. Such
estimates, which include infinite values for fixed effects and singular or
infinite variance components, can cause havoc to numerical estimation
procedures and inference. We introduce an appropriately scaled additive penalty
to the log-likelihood function, or an approximation thereof, which penalizes
the fixed effects by the Jeffreys' invariant prior for the model with no random
effects and the variance components by a composition of negative Huber loss
functions. The resulting maximum penalized likelihood estimates are shown to
lie in the interior of the parameter space. Appropriate scaling of the penalty
guarantees that the penalization is soft enough to preserve the optimal
asymptotic properties expected by the maximum likelihood estimator, namely
consistency, asymptotic normality, and Cram\'er-Rao efficiency. Our choice of
penalties and scaling factor preserves equivariance of the fixed effects
estimates under linear transformation of the model parameters, such as
contrasts. Maximum softly-penalized likelihood is compared to competing
approaches on two real-data examples, and through comprehensive simulation
studies that illustrate its superior finite sample performance.
