  The "fast iterative shrinkage-thresholding algorithm", a.k.a. FISTA, is one
of the most well-known first-order optimisation scheme in the literature, as it
achieves the worst-case $O(1/k^2)$ optimal convergence rate in terms of
objective function value. However, despite such an optimal theoretical
convergence rate, in practice the (local) oscillatory behaviour of FISTA often
damps its efficiency. Over the past years, various efforts are made in the
literature to improve the practical performance of FISTA, such as monotone
FISTA, restarting FISTA and backtracking strategies. In this paper, we propose
a simple yet effective modification to the original FISTA scheme which has two
advantages: it allows us to 1) prove the convergence of generated sequence; 2)
design a so-called "lazy-start" strategy which can up to an order faster than
the original scheme. Moreover, by exploring the properties of FISTA scheme, we
propose novel adaptive and greedy strategies which probes the limit of the
algorithm. The advantages of the proposed schemes are tested through problems
arising from inverse problem, machine learning and signal/image processing.
