  The fundamental celestial reference frame (CRF) is based on two catalogs of
astrometric positions, the third realization of the International Celestial
Reference Frame (ICRF3), and the much larger Gaia~CRF, built from the third
data release (DR3). The objects in common between these two catalogs are mostly
distant AGNs and quasars that are both sufficiently optically bright for Gaia
and radio-loud for the VLBI. This limited collection of reference objects is
crucially important for the mutual alignment of the two CRFs and maintenance of
all the other frames and coordinate systems branching from the ICRF. In this
paper, we show that the three components of ICRF3 (S/X, K, and X/Ka band
catalogs) have significantly different sky-correlated vector fields of position
offsets with respect to Gaia~DR3. When iteratively expanded in the vector
spherical harmonics up to degree 4 on a carefully vetted set of common sources,
each of these components includes several statistically significant terms. The
median sky-correlated offsets from the Gaia positions are found to be 56
$\mu$as for the S/X, 100 $\mu$as for the K, and 324 $\mu$as for the Ka
catalogs. The weighted mean vector field is subtracted from the Gaia reference
positions, while the deviations from that field are added to each of the ICRF3
components. The corrected positions from each of the four input catalogs are
combined into a single weighted mean catalog, which we propose to be the
current most accurate realization of an inertial radio-optical CRF.
