  In this paper, we study convex bi-level optimization problems where both the
inner and outer levels are given as a composite convex minimization. We propose
the Fast Bi-level Proximal Gradient (FBi-PG) algorithm, which can be
interpreted as applying FISTA to a dynamic regularized composite objective
function. The dynamic nature of the regularization parameters allows to achieve
an optimal fast convergence rate of $O(1/k^{2})$ in terms of the inner
objective function. This is the fastest known convergence rate under no
additional restrictive assumptions. We also show that FBi-PG achieves
sub-linear simultaneous rates in terms of both the inner and outer objective
functions. Moreover, we show that under an H\"olderian type error bound
assumption on the inner objective function, the FBi-PG algorithm converges to
an optimal solution of the bi-level optimization problem. Finally, we present
numerical experiments demonstrating the performances of the proposed scheme.
