  Deep neural networks have enhanced the performance of decision making systems
in many applications including image understanding, and further gains can be
achieved by constructing ensembles. However, designing an ensemble of deep
networks is often not very beneficial since the time needed to train the
networks is very high or the performance gain obtained is not very significant.
In this paper, we analyse error correcting output coding (ECOC) framework to be
used as an ensemble technique for deep networks and propose different design
strategies to address the accuracy-complexity trade-off. We carry out an
extensive comparative study between the introduced ECOC designs and the
state-of-the-art ensemble techniques such as ensemble averaging and gradient
boosting decision trees. Furthermore, we propose a combinatory technique which
is shown to achieve the highest classification performance amongst all.
