  Out-of-distribution (OOD) detection is crucial for the safe deployment of
neural networks. Existing CLIP-based approaches perform OOD detection by
devising novel scoring functions or sophisticated fine-tuning methods. In this
work, we propose SeTAR, a novel, training-free OOD detection method that
leverages selective low-rank approximation of weight matrices in
vision-language and vision-only models. SeTAR enhances OOD detection via
post-hoc modification of the model's weight matrices using a simple greedy
search algorithm. Based on SeTAR, we further propose SeTAR+FT, a fine-tuning
extension optimizing model performance for OOD detection tasks. Extensive
evaluations on ImageNet1K and Pascal-VOC benchmarks show SeTAR's superior
performance, reducing the false positive rate by up to 18.95% and 36.80%
compared to zero-shot and fine-tuning baselines. Ablation studies further
validate our approach's effectiveness, robustness, and generalizability across
different model backbones. Our work offers a scalable, efficient solution for
OOD detection, setting a new state-of-the-art in this area.
