  Low-light conditions not only hamper human visual experience but also degrade
the model's performance on downstream vision tasks. While existing works make
remarkable progress on day-night domain adaptation, they rely heavily on domain
knowledge derived from the task-specific nighttime dataset. This paper
challenges a more complicated scenario with border applicability, i.e.,
zero-shot day-night domain adaptation, which eliminates reliance on any
nighttime data. Unlike prior zero-shot adaptation approaches emphasizing either
image-level translation or model-level adaptation, we propose a similarity
min-max paradigm that considers them under a unified framework. On the image
level, we darken images towards minimum feature similarity to enlarge the
domain gap. Then on the model level, we maximize the feature similarity between
the darkened images and their normal-light counterparts for better model
adaptation. To the best of our knowledge, this work represents the pioneering
effort in jointly optimizing both aspects, resulting in a significant
improvement of model generalizability. Extensive experiments demonstrate our
method's effectiveness and broad applicability on various nighttime vision
tasks, including classification, semantic segmentation, visual place
recognition, and video action recognition. Code and pre-trained models are
available at https://red-fairy.github.io/ZeroShotDayNightDA-Webpage/.
