  This paper gives an in-depth review of the most common iterative methods for
unconstrained optimization using two functions that belong to a class of
Rosenbrock functions as a performance test. This study covers the Steepest
Gradient Descent Method, the Newton-Raphson Method, and the Fletcher-Reeves
Conjugate Gradient method. In addition, four different step-size selecting
methods including fixed-step-size, variable step-size, quadratic-fit, and
golden section method were considered. Due to the computational nature of
solving minimization problems, testing the algorithms is an essential part of
this paper. Therefore, an extensive set of numerical test results is also
provided to present an insightful and a comprehensive comparison of the
reviewed algorithms. This study highlights the differences and the trade-offs
involved in comparing these algorithms.
