  We model the effect of grain size distribution in a galaxy on the evolution
of CO and H$_2$ abundances. The formation and dissociation of CO and H$_2$ in
typical dense clouds are modelled in a manner consistent with the grain size
distribution. The evolution of grain size distribution is calculated based on
our previous model, which treats the galaxy as a one-zone object but includes
various dust processing mechanisms in the interstellar medium (ISM). We find
that typical dense clouds become fully molecular (H$_2$) when the dust surface
area increases by shattering while an increase of dust abundance by dust growth
in the ISM is necessary for a significant rise of the CO abundance.
Accordingly, the metallicity dependence of the CO-to-H$_2$ conversion factor,
$X_\mathrm{CO}$, is predominantly driven by dust growth. We also examine the
effect of grain size distribution in the galaxy by changing the dense gas
fraction, which controls the balance between coagulation and shattering,
clarifying that the difference in the grain size distribution significantly
affects $X_\mathrm{CO}$ even if the dust-to-gas ratio is the same. The star
formation time-scale, which controls the speed of metal enrichment also affects
the metallicity at which the CO abundance rapidly increases (or $X_\mathrm{CO}$
drops). We also propose dust-based formulae for $X_\mathrm{CO}$, which need
further tests for establishing their usefulness.
