  Measuring biodiversity is crucial for understanding ecosystem health. While
prior works have developed machine learning models for the taxonomic
classification of photographic images and DNA separately, in this work, we
introduce a multimodal approach combining both, using CLIP-style contrastive
learning to align images, DNA barcodes, and textual data in a unified embedding
space. This allows for accurate classification of both known and unknown insect
species without task-specific fine-tuning, leveraging contrastive learning for
the first time to fuse DNA and image data. Our method surpasses previous
single-modality approaches in accuracy by over 11% on zero-shot learning tasks,
showcasing its effectiveness in biodiversity studies.
