  Addressing the imperative need for efficient artificial intelligence in IoT
and edge computing, this study presents RepAct, a re-parameterizable adaptive
activation function tailored for optimizing lightweight neural networks within
the computational limitations of edge devices. By employing a multi-branch
structure with learnable adaptive weights, RepAct enriches feature processing
and enhances cross-layer interpretability. When evaluated on tasks such as
image classification and object detection, RepAct notably surpassed
conventional activation functions in lightweight networks, delivering up to a
7.92% accuracy boost on MobileNetV3-Small for the ImageNet100 dataset, while
maintaining computational complexity on par with HardSwish. This innovative
approach not only maximizes model parameter efficiency but also significantly
improves the performance and understanding capabilities of lightweight neural
networks, demonstrating its potential for real-time edge computing
applications.
