Deep Understanding of Bayesian Neural Networks and Evidential Deep Learning: A Synergistic Combination for Uncertainty Quantification

Today, as deep learning models are widely applied in high-risk domains such as medical diagnosis, autonomous driving, and financial risk control, it has become crucial for models to "know what they don't know". Traditional neural networks output probability distributions but cannot distinguish between aleatoric uncertainty (caused by data ambiguity) and epistemic uncertainty (caused by lack of model knowledge). The BNN-EDL project integrates Bayesian Neural Networks (BNNs) and Evidential Deep Learning (EDL) to provide a comprehensive uncertainty quantification solution for classification tasks, facilitating reliability assessment of deep learning models.

Take medical AI diagnosing rare diseases as an example: a 90% confidence level could stem from typical symptoms (clear data ) or lack of training data (insufficient model knowledge), which traditional models cannot distinguish. Aleatoric uncertainty arises from data noise/ambiguity (cannot be eliminated by adding more data), while epistemic uncertainty comes from model knowledge gaps (can be reduced by adding targeted data). Distinguishing between the two is critical for decision-making: high aleatoric uncertainty requires human intervention, and high epistemic uncertainty calls for targeted data collection.

The core of BNNs is to treat network weights as probability distributions instead of fixed values. During inference, the predictive marginal distribution is obtained by integrating over these distributions. The project uses two MCMC sampling methods: Stochastic Gradient Langevin Dynamics (SGLD) injects Gaussian noise into SGD to approximate posterior sampling; Stochastic Gradient Hamiltonian Monte Carlo (SGHMC) introduces momentum to efficiently explore the parameter space. A diagonal normal prior is used, combined with a classification likelihood function, and Bayesian Model Averaging (BMA) is applied to integrate predictions from multiple sampled networks, enhancing robustness and accuracy.

EDL treats network outputs as concentration parameters α (pseudo-counts) of Dirichlet distributions, which are naturally suited for representing uncertainty in classification probabilities. While softmax outputs sum to 1, EDL's α values reflect the model's "number of observations" for each class: large and similar α values → high confidence; small α values → uncertainty; one α much larger than others → strong preference. EDL learns the α parameters by maximizing the evidence (amount of data supporting the prediction).

BNNs alone have high sampling costs, while EDL alone struggles to capture model-level epistemic uncertainty. The hybrid architecture workflow: BNN sampling generates multiple network instances → each outputs Dirichlet parameters → BMA is applied to Dirichlet predictions, preserving EDL's predictive uncertainty modeling while introducing model uncertainty via BNN integration. Four configuration modes are supported: standard training + linear output (baseline), standard training + EDL output (pure evidential uncertainty), BNN training + linear output (pure Bayesian uncertainty), and BNN+EDL hybrid mode, which can be switched via YAML configuration.

The project implements multiple metrics: predictive entropy (total uncertainty, basis for rejecting predictions in decision-making), expected entropy (aleatoric uncertainty, data ambiguity), mutual information (epistemic uncertainty, model knowledge gaps), predictive variance (disagreement among ensemble models), and Expected Calibration Error (ECE, evaluating consistency between confidence and actual accuracy). Combining these metrics can diagnose model weaknesses and determine when to transfer decisions to human experts.

It supports MNIST/CIFAR-10 datasets and MLP/ResNet-20 (with filter response normalization) backbone networks; components are easily extensible via a registry system. Deployment recommendations: first build a standard training baseline, then gradually introduce EDL/BNN; set predictive entropy thresholds in high-risk scenarios to automatically transfer to human review; monitor mutual information to guide data collection. Future plans include deep integration with active learning, safe reinforcement learning, and explainable AI.

The BNN-EDL project provides a comprehensive and flexible uncertainty quantification framework: BNNs capture model uncertainty, EDL models predictive uncertainty, and their combination achieves a complete characterization. Today, as AI participates in critical decisions, this capability is a necessary condition for safe deployment. The project's modular design lays the foundation for future extensions and is crucial for building reliable and trustworthy AI systems.