  The increasing deployment of artificial intelligence (AI) for critical
decision-making amplifies the necessity for trustworthy AI, where uncertainty
estimation plays a pivotal role in ensuring trustworthiness. Dropout-based
Bayesian Neural Networks (BayesNNs) are prominent in this field, offering
reliable uncertainty estimates. Despite their effectiveness, existing
dropout-based BayesNNs typically employ a uniform dropout design across
different layers, leading to suboptimal performance. Moreover, as diverse
applications require tailored dropout strategies for optimal performance,
manually optimizing dropout configurations for various applications is both
error-prone and labor-intensive. To address these challenges, this paper
proposes a novel neural dropout search framework that automatically optimizes
both the dropout-based BayesNNs and their hardware implementations on FPGA. We
leverage one-shot supernet training with an evolutionary algorithm for
efficient dropout optimization. A layer-wise dropout search space is introduced
to enable the automatic design of dropout-based BayesNNs with heterogeneous
dropout configurations. Extensive experiments demonstrate that our proposed
framework can effectively find design configurations on the Pareto frontier.
Compared to manually-designed dropout-based BayesNNs on GPU, our search
approach produces FPGA designs that can achieve up to 33X higher energy
efficiency. Compared to state-of-the-art FPGA designs of BayesNN, the solutions
from our approach can achieve higher algorithmic performance and energy
efficiency.
