  We solve the mean-field-like $p$-spin Ising model under a spatio-temporal
inhomogeneous transverse field to study the effects of inhomogeneity on the
performance of quantum annealing. We find that the problematic first-order
quantum phase transition that arises under the conventional homogeneous field
protocol can be avoided if the temperature is zero and the local field is
completely turned off site by site after a finite time. When these ideal
conditions are not satisfied, a new series of first-order transitions appear,
which prevents us from driving the system while avoiding first-order
transitions. Nevertheless, under these non-ideal conditions, quantitative
improvements can be obtained in terms of narrower tunneling barriers in the
free energy landscape. A comparison with classical simulated annealing
establishes a limited quantum advantage in the ideal case, since inhomogeneous
temperature driving in simulated annealing cannot remove a first-order
transition, in contrast to the quantum case. The classical model of spin-vector
Monte Carlo is also analyzed, and we find it to have the same thermodynamic
phase diagram as the quantum model in the ideal case, with deviations arising
at non-zero temperature.
