  Predicting legal judgments with reliable confidence is paramount for
responsible legal AI applications. While transformer-based deep neural networks
(DNNs) like BERT have demonstrated promise in legal tasks, accurately assessing
their prediction confidence remains crucial. We present a novel Bayesian
approach called BayesJudge that harnesses the synergy between deep learning and
deep Gaussian Processes to quantify uncertainty through Bayesian kernel Monte
Carlo dropout. Our method leverages informative priors and flexible data
modelling via kernels, surpassing existing methods in both predictive accuracy
and confidence estimation as indicated through brier score. Extensive
evaluations of public legal datasets showcase our model's superior performance
across diverse tasks. We also introduce an optimal solution to automate the
scrutiny of unreliable predictions, resulting in a significant increase in the
accuracy of the model's predictions by up to 27\%. By empowering judges and
legal professionals with more reliable information, our work paves the way for
trustworthy and transparent legal AI applications that facilitate informed
decisions grounded in both knowledge and quantified uncertainty.
