  The recent advent of `Internet of Things' (IOT) has increased the demand for
enabling AI-based edge computing. This has necessitated the search for
efficient implementations of neural networks in terms of both computations and
storage. Although extreme quantization has proven to be a powerful tool to
achieve significant compression over full-precision networks, it can result in
significant degradation in performance. In this work, we propose extremely
quantized hybrid network architectures with both binary and full-precision
sections to emulate the classification performance of full-precision networks
while ensuring significant energy efficiency and memory compression. We explore
several hybrid network architectures and analyze the performance of the
networks in terms of accuracy, energy efficiency and memory compression. We
perform our analysis on ResNet and VGG network architectures. Among the
proposed network architectures, we show that the hybrid networks with
full-precision residual connections emerge as the optimum by attaining
accuracies close to full-precision networks while achieving excellent memory
compression, up to 21.8x in case of VGG-19. This work demonstrates an effective
way of hybridizing networks which achieve performance close to full-precision
networks while attaining significant compression, furthering the feasibility of
using such networks for energy-efficient neural computing in IOT-based edge
devices.
