  Convolutional Neural Networks (CNN) have revolutionized perception for color
images, and their application to sonar images has also obtained good results.
But in general CNNs are difficult to train without a large dataset, need manual
tuning of a considerable number of hyperparameters, and require many careful
decisions by a designer. In this work, we evaluate three common decisions that
need to be made by a CNN designer, namely the performance of transfer learning,
the effect of object/image size and the relation between training set size. We
evaluate three CNN models, namely one based on LeNet, and two based on the Fire
module from SqueezeNet. Our findings are: Transfer learning with an SVM works
very well, even when the train and transfer sets have no classes in common, and
high classification performance can be obtained even when the target dataset is
small. The ADAM optimizer combined with Batch Normalization can make a high
accuracy CNN classifier, even with small image sizes (16 pixels). At least 50
samples per class are required to obtain $90\%$ test accuracy, and using
Dropout with a small dataset helps improve performance, but Batch Normalization
is better when a large dataset is available.
