  While developing new unsupervised domain translation methods for endoscopy
videos, it is typical to start with approaches that initially work for
individual frames without temporal consistency. Once an individual-frame model
has been finalized, additional contiguous frames are added with a modified deep
learning architecture to train a new model for temporal consistency. This
transition to temporally-consistent deep learning models, however, requires
significantly more computational and memory resources for training. In this
paper, we present a lightweight solution with a tunable temporal parameter,
RT-GAN (Recurrent Temporal GAN), for adding temporal consistency to individual
frame-based approaches that reduces training requirements by a factor of 5. We
demonstrate the effectiveness of our approach on two challenging use cases in
colonoscopy: haustral fold segmentation (indicative of missed surface) and
realistic colonoscopy simulator video generation. The datasets, accompanying
code, and pretrained models will be made available at
\url{https://github.com/nadeemlab/CEP}.
