  [Abridged] We extend color-magnitude relations for moderate-luminosity X-ray
AGN hosts and non-AGN galaxies through the galaxy formation epoch in the
Chandra Deep Fields. We utilized analyses of color-magnitude diagrams (CMDs) to
assess the role of moderate-luminosity AGNs in galaxy evolution. First, we
confirm some previous results and extend them to higher redshifts, e.g., there
is no apparent color bimodality for AGN hosts from z~0-2, but non-AGN galaxy
color bimodality exists up to z~3; most AGNs reside in massive hosts and the
AGN fraction rises strongly toward higher stellar mass up to z~2-3; and the
colors of both AGN hosts and non-AGN galaxies become redder as the stellar mass
increases up to z~2-3. Second, we point out that it is critical to use
mass-matched samples to examine color-magnitude relations of AGN hosts and
non-AGN galaxies. We show that for mass-matched samples up to z~2-3, AGN hosts
lie in the same CMD region as non-AGN galaxies; i.e., there is no specific
clustering of AGN hosts around the red sequence, the top of the blue cloud, or
the green valley in between. The AGN fraction (~10%) is mostly independent of
host-galaxy color, providing an indication of the duty cycle of SMBH growth in
typical massive galaxies. These results are in contrast to those obtained with
non-mass-matched samples where there is apparent AGN clustering in the CMD and
the AGN fraction generally increases as the color becomes redder. We also find,
for mass-matched samples, that the SFRs of AGN hosts are typically a factor of
~2-3 larger than those of non-AGN galaxies at z~0-1, whereas this difference
diminishes at z~1-3. Our results can be reasonably explained by two main
ingredients, color-mass correlation and passive or secular evolution of
galaxies; and thus tightly constrain any effects from moderate-luminosity AGN
feedback upon color-magnitude properties over the ~80% of cosmic time.
