3 共轭斜量法 （ Conjugate Gradient ） Modern optimization methods : “conjugate direction” methods. A method to solve quadratic function minimization: (A is symmetric and positive definite)
Conjugate Gradient Originally aimed to solve linear problems: Later extended to general functions under rational of quadratic approximation to a function is quite accurate.
Conjugate Gradient The basic idea: decompose the n-dimensional quadratic problem into n problems of 1-dimension This is done by exploring the function in “conjugate directions”. Definition: A-conjugate vectors:
Conjugate Gradient If there is an A-conjugate basis then: N problems in 1-dimension (simple smiling quadratic) The global minimizer is calculated sequentially starting from x 0 :