  A detailed study of two-photon exchange in unpolarized and polarized elastic
electron--nucleon scattering is presented, taking particular account of nucleon
finite size effects. Contributions from nucleon elastic intermediate states are
found to have a strong angular dependence, which leads to a partial resolution
of the discrepancy between the Rosenbluth and polarization transfer
measurements of the proton electric to magnetic form factor ratio, G_E/G_M. The
two-photon exchange contribution to the longitudinal polarization transfer P_L
is small, whereas the contribution to the transverse polarization transfer P_T
is enhanced at backward angles by several percent, increasing with Q^2. This
gives rise to a small, ~3% suppression of G_E/G_M obtained from the
polarization transfer ratio P_T/P_L at large Q^2. We also compare the
two-photon exchange effects with data on the ratio of e^+ p to e^- p cross
sections, which is predicted to be enhanced at backward angles. Finally, we
evaluate the corrections to the form factors of the neutron, and estimate the
elastic intermediate state contribution to the ^3He form factors.
