  The apparent distribution of large-scale structures in the universe is
sensitive to the velocity/potential of the sources as well as the potential
along the line-of-sight through the mapping from real space to redshift space
(redshift-space distortions, RSD). Since odd multipoles of the halo
cross-correlation function vanish when considering standard Doppler RSD, the
dipole is a sensitive probe of relativistic and wide-angle effects. We build a
catalogue of ten million haloes (Milky-Way size to galaxy-cluster size) from
the full-sky light-cone of a new "RayGalGroupSims" N-body simulation which
covers a volume of ($2.625~h^{-1}$Gpc)$^3$ with $4096^3$ particles. Using
ray-tracing techniques, we find the null geodesics connecting all the sources
to the observer. We then self-consistently derive all the relativistic
contributions (in the weak-field approximation) to RSD: Doppler, transverse
Doppler, gravitational, lensing and integrated Sachs-Wolfe. It allows us, for
the first time, to disentangle all contributions to the dipole from linear to
non-linear scales. At large scale, we recover the linear predictions dominated
by a contribution from the divergence of neighbouring line-of-sights. While the
linear theory remains a reasonable approximation of the velocity contribution
to the dipole at non-linear scales it fails to reproduce the potential
contribution below $30-60~h^{-1}$Mpc (depending on the halo mass). At scales
smaller than $\sim 10~h^{-1}$Mpc, the dipole is dominated by the asymmetry
caused by the gravitational redshift. The transition between the two regimes is
mass dependent as well. We also identify a new non-trivial contribution from
the non-linear coupling between potential and velocity terms.
