  Computing the regularized solution of Bayesian linear inverse problems as
well as the corresponding regularization parameter is highly desirable in many
applications. This paper proposes a novel iterative method, termed the
Projected Newton method (PNT), that can simultaneously update the
regularization parameter and solution step by step without requiring any
high-cost matrix inversions or decompositions. By reformulating the Tikhonov
regularization as a constrained minimization problem and writing its Lagrangian
function, a Newton-type method coupled with a Krylov subspace method, called
the generalized Golub-Kahan bidiagonalization, is employed for the
unconstrained Lagrangian function. The resulting PNT algorithm only needs
solving a small-scale linear system to get a descent direction of a merit
function at each iteration, thus significantly reducing computational overhead.
Rigorous convergence results are proved, showing that PNT always converges to
the unique regularized solution and the corresponding Lagrangian multiplier.
Experimental results on both small and large-scale Bayesian inverse problems
demonstrate its excellent convergence property, robustness and efficiency.
Given that the most demanding computational tasks in PNT are primarily
matrix-vector products, it is particularly well-suited for large-scale
problems.
