  We focus on the solutions of second-order stable linear difference equations
and demonstrate that their behavior can be non-monotone and exhibit peak
effects depending on initial conditions. The results are applied to the
analysis of the accelerated unconstrained optimization method -- the Heavy Ball
method. We explain non-standard behavior of the method discovered in practical
applications. In addition, such non-monotonicity complicates the correct choice
of the parameters in optimization methods. We propose to overcome this
difficulty by introducing new Lyapunov function which should decrease
monotonically. By use of this function convergence of the method is established
under less restrictive assumptions (for instance, with the lack of convexity).
We also suggest some restart techniques to speed up the methods' convergence.
