  Heavy-Ball method (HB) is known for its simplicity in implementation and
practical efficiency. However, as with other momentum methods, it has
non-monotone behavior, and for optimal parameters, the method suffers from the
so-called peak effect. To address this issue, in this paper, we consider an
averaged version of Heavy-Ball method (AHB). We show that for quadratic
problems AHB has a smaller maximal deviation from the solution than HB.
Moreover, for general convex and strongly convex functions, we prove
non-accelerated rates of global convergence of AHB and its weighted version. We
conduct several numerical experiments on minimizing quadratic and non-quadratic
functions to demonstrate the advantages of using averaging for HB.
