  We propose a method to determine the effective potential of QCD from the gap
equation, by introducing the homotopy method between the solutions of the
equation of motion. Via this method, the effective potential beyond the bare
vertex approximation is obtained, which then generalizes the Cornwall, Jackiw
and Tomboulis (CJT) effective potential for the bilocal composite operators.
Moreover, the extended effective potential is set to be a function of self
energy instead of the propagator, which is the key point for the potential to
be bounded from below. We then investigate the extended effective potential in
the cases of phase transition of the QCD vacuum with a small current quark
mass, and the first-order phase transition of QCD at finite temperature and
high baryon chemical potential. In the former case, the effective potential
shows as an inflection point at the critical mass where the multiple solutions
of the Dyson-Schwinger equation (DSE) vanishes, which is consistent with that
obtained by solving the DSE directly. For the latter case, the in-medium
properties, such as the latent heat and the difference of trace anomaly, of QCD
is obtained.
