  The bismuth-halide Bi$_4$I$_4$ undergoes a structural transition around
$T_P\sim 300$K, which separates a high-temperature $\beta$ phase ($T>T_P$) from
a low-temperature $\alpha$ phase ($T<T_P$). $\alpha$ and $\beta$ phases are
suggested to host electronic band structures with distinct topological
classifications. Rapid quenching was reported to stabilize a metastable
$\beta$-Bi$_4$I$_4$ at $T<T_P$, making possible a comparative study of the
physical properties of the two phases in the same low-temperature range. In
this work, we present a structural investigation of the Bi$_4$I$_4$ before and
after quenching together with electrical resistivity measurements. We found
that rapid cooling does not consistently lead to a metastable
$\beta$-Bi$_4$I$_4$, and a quick transition to $\alpha$-Bi$_4$I$_4$ is
observed. As a result, the comparison of putative signatures of different
topologies attributed to a specific structural phase should be carefully
considered. The observed phase instability is accompanied by an increase in
iodine vacancies and by a change in the temperature dependence of electrical
resistivity, pointing to native defects as a possible origin of our finding.
Density functional theory (DFT) calculations support the scenario that iodine
vacancies, together with bismuth antisites and interstitials, are among the
defects that are more likely to occur in Bi$_4$I$_4$ during the growth.
