  Even though the Convolutional Neural Networks (CNN) has shown superior
results in the field of computer vision, it is still a challenging task to
implement computer vision algorithms in real-time at the edge, especially using
a low-cost IoT device due to high memory consumption and computation
complexities in a CNN. Network compression methodologies such as weight
pruning, filter pruning, and quantization are used to overcome the above
mentioned problem. Even though filter pruning methodology has shown better
performances compared to other techniques, irregularity of the number of
filters pruned across different layers of a CNN might not comply with majority
of the neural computing hardware architectures. In this paper, a novel greedy
approach called cluster pruning has been proposed, which provides a structured
way of removing filters in a CNN by considering the importance of filters and
the underlying hardware architecture. The proposed methodology is compared with
the conventional filter pruning algorithm on Pascal-VOC open dataset, and
Head-Counting dataset, which is our own dataset developed to detect and count
people entering a room. We benchmark our proposed method on three hardware
architectures, namely CPU, GPU, and Intel Movidius Neural Computer Stick (NCS)
using the popular SSD-MobileNet and SSD-SqueezeNet neural network architectures
used for edge-AI vision applications. Results demonstrate that our method
outperforms the conventional filter pruning methodology, using both datasets on
above mentioned hardware architectures. Furthermore, a low cost IoT hardware
setup consisting of an Intel Movidius-NCS is proposed to deploy an edge-AI
application using our proposed pruning methodology.
