  Recently, progress has been made in the understanding of anomalous
vibrational excitations in amorphous solids. In the lowest-frequency region,
the vibrational spectrum follows a non-Debye quartic law, which persists up to
zero frequency without any frequency gap. This gapless vibrational density of
states (vDOS) suggests that glasses are on the verge of instability. This
feature of marginal stability is now highlighted as a key concept in the
theories of glasses. In particular, the elasticity theory based on marginal
stability predicts the gapless vDOS. However, this theory yields a quadratic
law and \textit{not} the quartic law. To address this inconsistency, we
presented a new type of instability, which is different from the conventional
one, and proposed that amorphous solids are marginally stable considering the
new instability in the preceding study~[M. Shimada, H. Mizuno, and A. Ikeda,
Soft Matter, {\bf 16}, 7279, 2020]. In this study, we further extend and detail
the results for these instabilities. By analyzing various examples of disorder,
we demonstrate that real glasses in finite spatial dimensions can be marginally
stable by the proposed novel instability.
