  We introduce a notion of self-concordant smoothing for minimizing the sum of
two convex functions, one of which is smooth and the other may be nonsmooth.
The key highlight of our approach is in a natural property of the resulting
problem's structure which provides us with a variable-metric selection method
and a step-length selection rule particularly suitable for proximal Newton-type
algorithms. In addition, we efficiently handle specific structures promoted by
the nonsmooth function, such as $\ell_1$-regularization and group-lasso
penalties. We prove the convergence of two resulting algorithms: Prox-N-SCORE,
a proximal Newton algorithm and Prox-GGN-SCORE, a proximal generalized
Gauss-Newton algorithm. The Prox-GGN-SCORE algorithm highlights an important
approximation procedure which helps to significantly reduce most of the
computational overhead associated with the inverse Hessian. This approximation
is essentially useful for overparameterized machine learning models and in the
mini-batch settings. Numerical examples on both synthetic and real datasets
demonstrate the efficiency of our approach and its superiority over existing
approaches. A Julia package implementing the proposed algorithms is available
at https://github.com/adeyemiadeoye/SelfConcordantSmoothOptimization.jl.
