  Quantum annealing (QA) is one of the methods to solve combinatorial
optimization problems. We can estimate a computational time of QA by using the
so-called adiabatic condition derived from the adiabatic theorem. The adiabatic
condition consists of two parts: an energy gap and a transition matrix. It is
believed that the computational time mainly depends on the energy gap during QA
and is inversely proportional to a polynomial of the minimal energy gap. In
this paper, we challenge this common wisdom. We propose a general method to
construct counterintuitive models with a constant energy gap during QA where QA
with a constant annealing time fails despite a constant energy gap. In our
formalism, we choose a known model exhibiting an exponentially small energy gap
during QA, and we modify the model by adding a specific penalty term to the
Hamiltonian. In the modified model, the transition matrix in the adiabatic
condition becomes exponentially large with the number of qubits, while the
energy gap remains constant. As concrete examples we consider the adiabatic
Grover search and the ferromagnetic p-spin model. In these cases, by adding the
penalty term, the success probability of QA in the modified models become
exponentially small despite a constant energy gap. Our results paves a way for
better understanding of the QA performance.
