  We use a sample of 37 of the densest clusters and protoclusters across $1.3
\le z \le 3.2$ from the Clusters Around Radio-Loud AGN (CARLA) survey to study
the formation of massive cluster galaxies. We use optical $i'$-band and
infrared 3.6$\mu$m and 4.5$\mu$m images to statistically select sources within
these protoclusters and measure their median observed colours; $\langle
i'-[3.6] \rangle$. We find the abundance of massive galaxies within the
protoclusters increases with decreasing redshift, suggesting these objects may
form an evolutionary sequence, with the lower redshift clusters in the sample
having similar properties to the descendants of the high redshift
protoclusters. We find that the protocluster galaxies have an approximately
unevolving observed-frame $i'-[3.6]$ colour across the examined redshift range.
We compare the evolution of the $\langle i'-[3.6] \rangle$ colour of massive
cluster galaxies with simplistic galaxy formation models. Taking the full
cluster population into account, we show that the formation of stars within the
majority of massive cluster galaxies occurs over at least 2Gyr, and peaks at $z
\sim 2$-3. From the median $i'-[3.6]$ colours we cannot determine the star
formation histories of individual galaxies, but their star formation must have
been rapidly terminated to produce the observed red colours. Finally, we show
that massive galaxies at $z>2$ must have assembled within 0.5Gyr of them
forming a significant fraction of their stars. This means that few massive
galaxies in $z>2$ protoclusters could have formed via dry mergers.
