  We measure the merger fraction of massive galaxies using the
UltraVISTA/COSMOS $Ks$-band selected catalog, complemented with the deeper,
higher resolution 3DHST+CANDELS catalog selected in the HST/WFC3 $H$-band,
presenting the largest mass-complete photometric merger sample up to $z\sim3$.
We find that selecting mergers using the $H_{160}$-band flux ratio leads to an
increasing merger fraction with redshift, while selecting mergers using the
stellar mass ratio causes a diminishing redshift dependence. Defining major and
minor mergers as having stellar mass ratios of 1:1 - 4:1 and 4:1 - 10:1
respectively, the results imply $\sim$1 major and $\lesssim$1 minor merger for
an average massive (log$(M_{\star}/M_{\odot}) \geqslant 10.8$) galaxy during
$z=0.1-2.5$. There may be an additional $\sim 0.5(0.3)$ major (minor) merger if
we use the $H$-band flux ratio selection. The observed amount of major merging
alone is sufficient to explain the observed number density evolution for the
very massive (log$(M_{\star}/M_{\odot}) \geqslant 11.1$) galaxies. We argue
that these very massive galaxies can put on a maximum of $6\%$ of stellar mass
in addition to major and minor merging, so that their number density evolution
remains consistent with observations. The observed number of major and minor
mergers can increase the size of a massive quiescent galaxy by a factor of two
at most. This amount of merging is enough to bring the compact quiescent
galaxies formed at $z>2$ to lie at $1\sigma$ below the mean of the stellar
mass-size relation as measured in some works (e.g. Newman et al. 2012), but
additional mechanisms are needed to fully explain the evolution, and to be
consistent with works suggesting stronger evolution (e.g. van der Wel et al.
2014).
