  This paper strives to recognize activities in the dark, as well as in the
day. We first establish that state-of-the-art activity recognizers are
effective during the day, but not trustworthy in the dark. The main causes are
the limited availability of labeled dark videos to learn from, as well as the
distribution shift towards the lower color contrast at test-time. To compensate
for the lack of labeled dark videos, we introduce a pseudo-supervised learning
scheme, which utilizes easy to obtain unlabeled and task-irrelevant dark videos
to improve an activity recognizer in low light. As the lower color contrast
results in visual information loss, we further propose to incorporate the
complementary activity information within audio, which is invariant to
illumination. Since the usefulness of audio and visual features differs
depending on the amount of illumination, we introduce our `darkness-adaptive'
audio-visual recognizer. Experiments on EPIC-Kitchens, Kinetics-Sound, and
Charades demonstrate our proposals are superior to image enhancement, domain
adaptation and alternative audio-visual fusion methods, and can even improve
robustness to local darkness caused by occlusions. Project page:
https://xiaobai1217.github.io/Day2Dark/
