  Visual analysis of complex fish habitats is an important step towards
sustainable fisheries for human consumption and environmental protection. Deep
Learning methods have shown great promise for scene analysis when trained on
large-scale datasets. However, current datasets for fish analysis tend to focus
on the classification task within constrained, plain environments which do not
capture the complexity of underwater fish habitats. To address this limitation,
we present DeepFish as a benchmark suite with a large-scale dataset to train
and test methods for several computer vision tasks. The dataset consists of
approximately 40 thousand images collected underwater from 20 \green{habitats
in the} marine-environments of tropical Australia. The dataset originally
contained only classification labels. Thus, we collected point-level and
segmentation labels to have a more comprehensive fish analysis benchmark. These
labels enable models to learn to automatically monitor fish count, identify
their locations, and estimate their sizes. Our experiments provide an in-depth
analysis of the dataset characteristics, and the performance evaluation of
several state-of-the-art approaches based on our benchmark. Although models
pre-trained on ImageNet have successfully performed on this benchmark, there is
still room for improvement. Therefore, this benchmark serves as a testbed to
motivate further development in this challenging domain of underwater computer
vision. Code is available at: https://github.com/alzayats/DeepFish
