  Quantifying the contribution of mergers to the stellar mass of galaxies is
key for constraining the mechanisms of galaxy assembly across cosmic time.
However, the mapping between observable galaxy properties and merger histories
is not trivial: cosmological galaxy simulations are the only tools we have for
calibration. We study the robustness of a simulation-based inference of the
ex-situ stellar mass fraction of nearby galaxies to different observables --
integrated and spatially-resolved -- and to different galaxy formation models
-- IllustrisTNG and EAGLE -- with Machine Learning. We find that at fixed
simulation, the fraction of accreted stars can be inferred with very high
accuracy, with an error $\sim5$ per cent (10 per cent) from 2D integral-field
spectroscopic maps (integrated quantities) throughout the considered stellar
mass range. A bias (> 5 per cent) and an increase in scatter by a factor of 2
are introduced when testing with a different simulation, revealing a lack of
generalization to distinct galaxy-formation models. Interestingly, upon using
only stellar mass and kinematics maps in the central galactic regions for
training, we find that this bias is removed and the ex-situ stellar mass
fraction can be recovered in both simulations with < 15 per cent scatter,
independently of the training set's origin. This opens up the door to a
potential robust inference of the accretion histories of galaxies from existing
Integral Field Unit surveys, such as MaNGA, covering a similar field of view
(FOV) and containing spatially-resolved spectra for tens of thousands of nearby
galaxies.
