  Fish feeding intensity assessment (FFIA) aims to evaluate fish appetite
changes during feeding, which is crucial in industrial aquaculture
applications. Existing FFIA methods are limited by their robustness to noise,
computational complexity, and the lack of public datasets for developing the
models. To address these issues, we first introduce AV-FFIA, a new dataset
containing 27,000 labeled audio and video clips that capture different levels
of fish feeding intensity. Then, we introduce multi-modal approaches for FFIA
by leveraging the models pre-trained on individual modalities and fused with
data fusion methods. We perform benchmark studies of these methods on AV-FFIA,
and demonstrate the advantages of the multi-modal approach over the
single-modality based approach, especially in noisy environments. However,
compared to the methods developed for individual modalities, the multimodal
approaches may involve higher computational costs due to the need for
independent encoders for each modality. To overcome this issue, we further
present a novel unified mixed-modality based method for FFIA, termed as U-FFIA.
U-FFIA is a single model capable of processing audio, visual, or audio-visual
modalities, by leveraging modality dropout during training and knowledge
distillation using the models pre-trained with data from single modality. We
demonstrate that U-FFIA can achieve performance better than or on par with the
state-of-the-art modality-specific FFIA models, with significantly lower
computational overhead, enabling robust and efficient FFIA for improved
aquaculture management.
