  Logistic regression training over encrypted data has been an attractive idea
to security concerns for years. In this paper, we propose a faster gradient
variant called $\texttt{quadratic gradient}$ for privacy-preserving logistic
regression training. The core of $\texttt{quadratic gradient}$ can be seen as
an extension of the simplified fixed Hessian. We enhance Nesterov's accelerated
gradient (NAG) and Adaptive Gradient Algorithm (Adagrad) respectively with
$\texttt{quadratic gradient}$ and evaluate the enhanced algorithms on several
datasets. Experiments show that the enhanced methods have a state-of-the-art
performance in convergence speed compared to the raw first-order gradient
methods. We then adopt the enhanced NAG method to implement homomorphic
logistic regression training, obtaining a comparable result by only $3$
iterations. There is a promising chance that $\texttt{quadratic gradient}$
could be used to enhance other first-order gradient methods for general
numerical optimization problems.
