  Several materials in the regime of strong spin-orbit interaction such as
HgTe, the pyrochlore iridate Pr$_2$Ir$_2$O$_7$, and the half-Heusler compound
LaPtBi, as well as various systems related to these three prototype materials,
are believed to host a quadratic band touching point at the Fermi level.
Recently, it has been proposed that such a three-dimensional gapless state is
unstable to a Mott-insulating ground state at low temperatures when the number
of band touching points $N$ at the Fermi level is smaller than a certain
critical number $N_c$. We further substantiate and quantify this scenario by
various approaches. Using $\epsilon$ expansion near two spatial dimensions, we
show that $N_c = 64/(25 \epsilon^2) + O(1/\epsilon)$ and demonstrate that the
instability for $N < N_c$ is towards a nematic ground state that can be
understood as if the system were under (dynamically generated) uniaxial strain.
We also propose a truncation of the functional renormalization group equations
in the dynamical bosonization scheme which we show to agree to one-loop order
with the results from $\epsilon$ expansion both near two as well as near four
dimensions, and which smoothly interpolates between these two perturbatively
accessible limits for general $2<d<4$. Directly in $d=3$ we therewith find $N_c
= 1.86$, and thus again above the physical $N=1$. All these results are
consistent with the prediction that the interacting ground state of pure,
unstrained HgTe, and possibly also Pr$_2$Ir$_2$O$_7$, is a strong topological
insulator with a dynamically-generated gap -- a topological Mott insulator.
