  We develop a projection operator formalism for studying both the zero
temperature equilibrium phase diagram and the non-equilibrium dynamics of the
Bose-Hubbard model. Our work, which constitutes an extension of Phys. Rev.
Lett. {\bf 106}, 095702 (2011), shows that the method provides an accurate
description of the equilibrium zero temperature phase diagram of the
Bose-Hubbard model for several lattices in two- and three-dimensions (2D and
3D). We show that the accuracy of this method increases with the coordination
number $z_0$ of the lattice and reaches to within 0.5% of quantum Monte Carlo
data for lattices with $z_0=6$. We compute the excitation spectra of the bosons
using this method in the Mott and the superfluid phases and compare our results
with mean-field theory. We also show that the same method may be used to
analyze the non-equilibrium dynamics of the model both in the Mott phase and
near the superfluid-insulator quantum critical point where the hopping
amplitude $J$ and the on-site interaction $U$ satisfy $z_0J/U \ll 1$. In
particular, we study the non-equilibrium dynamics of the model both subsequent
to a sudden quench of the hopping amplitude $J$ and during a ramp from $J_i$ to
$J_f$ characterized by a ramp time $\tau$ and exponent $\alpha$: $J(t)=J_i
+(J_f-J_i) (t/\tau)^{\alpha}$. We compute the wavefunction overlap $F$, the
residual energy $Q$, the superfluid order parameter $\Delta(t)$, the equal-time
order parameter correlation function $C(t)$, and the defect formation
probability $P$ for the above-mentioned protocols and provide a comparison of
our results to their mean-field counterparts. We find that $Q$, $F$, and $P$ do
not exhibit the expected universal scaling. We explain this absence of
universality and show that our results for linear ramps compare well with the
recent experimental observations.
