  Abridged. We aim at exploring to which extent the remnant keeps memory of the
asymmetries that develop stochastically in the neutrino-heating layer due to
hydrodynamic instabilities (e.g., convective overturn and the standing
accretion shock instability) during the first second after core bounce. We
coupled a 3D HD model of a neutrino-driven SN explosion with 3D MHD/HD
simulations of the remnant formation. The simulations cover 2000 years of
expansion and include all physical processes relevant to describe the
complexities in the SN evolution and the subsequent interaction of the stellar
debris with the wind of the progenitor star. The interaction of large-scale
asymmetries left from the earliest phases of the explosion with the reverse
shock produces, at the age of $\approx 350$~years, an ejecta structure and a
remnant morphology which are remarkably similar to those observed in Cas A.
Small-scale structures in the large-scale Fe-rich plumes created during the
initial stages of the SN, combined with HD instabilities that develop after the
passage of the reverse shock, naturally produce a pattern of ring- and
crown-like structures of shocked ejecta. The consequence is a spatial inversion
of the ejecta layers with Si-rich ejecta being physically interior to Fe-rich
ejecta. The full-fledged remnant shows voids and cavities in the innermost
unshocked ejecta resulting from the expansion of Fe-rich plumes and their
inflation due to the decay of radioactive species. The asymmetric distributions
of $^{44}$Ti and $^{56}$Fe and their abundance ratio are both compatible with
those inferred from high-energy observations of Chandra and NuSTAR. The main
asymmetries observed in the ejecta distribution of Cas A can be explained by
the interaction of the reverse shock with the large-scale asymmetries that
developed from stochastic processes that originate during the first seconds of
the SN blast.
