  Most recently, with the proliferation of IoT devices, computational nodes in
manufacturing systems IIoT(Industrial-Internet-of-things) and the lunch of 5G
networks, there will be millions of connected devices generating a massive
amount of data. In such an environment, the controlling systems need to be
intelligent enough to deal with a vast amount of data to detect defects in a
real-time process. Driven by such a need, artificial intelligence models such
as deep learning have to be deployed into IIoT systems. However, learning and
using deep learning models are computationally expensive, so an IoT device with
limited computational power could not run such models. To tackle this issue,
edge intelligence had emerged as a new paradigm towards running Artificial
Intelligence models on edge devices. Although a considerable amount of studies
have been proposed in this area, the research is still in the early stages. In
this paper, we propose a novel edge-based multi-phase pruning pipelines to
ensemble learning on IIoT devices. In the first phase, we generate a diverse
ensemble of pruned models, then we apply integer quantisation, next we prune
the generated ensemble using a clustering-based technique. Finally, we choose
the best representative from each generated cluster to be deployed to a
distributed IoT environment. On CIFAR-100 and CIFAR-10, our proposed approach
was able to outperform the predictability levels of a baseline model (up to
7%), more importantly, the generated learners have small sizes (up to 90%
reduction in the model size) that minimise the required computational
capabilities to make an inference on the resource-constraint devices.
