  Many machine learning models depend on solving a large scale optimization
problem. Recently, sub-sampled Newton methods have emerged to attract much
attention for optimization due to their efficiency at each iteration, rectified
a weakness in the ordinary Newton method of suffering a high cost at each
iteration while commanding a high convergence rate. In this work we propose two
new efficient Newton-type methods, Refined Sub-sampled Newton and Refined
Sketch Newton. Our methods exhibit a great advantage over existing sub-sampled
Newton methods, especially when Hessian-vector multiplication can be calculated
efficiently. Specifically, the proposed methods are shown to converge
superlinearly in general case and quadratically under a little stronger
assumption. The proposed methods can be generalized to a unifying framework for
the convergence proof of several existing sub-sampled Newton methods, revealing
new convergence properties. Finally, we empirically evaluate the performance of
our methods on several standard datasets and the results show consistent
improvement in computational efficiency.
