Jasper Snoek, Michael Dusenberry, Ghassen Jerfel, Yeming Wen, Yi-An Ma, Katherine Heller, Balaji Lakshminarayanan, Dustin Tran
We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Bayesian neural networks (BNNs) demonstrate promising success in improving the robustness and uncertainty quantification of modern deep learning. However, they generally struggle with underfitting at scale and parameter efficiency. On the other hand, deep ensembles have emerged as alternatives for uncertainty quantification that, while outperforming BNNs on certain problems, also suffer from efficiency issues. It remains unclear how to combine the strengths of these two approaches and remediate their common issues. To tackle this challenge, we propose a rank-1 parameterization of BNNs, where each weight matrix involves only a distribution on a rank-1 subspace. We also revisit the use of mixture approximate posteriors to capture multiple modes, where unlike typical mixtures, this approach admits a significantly smaller memory increase (e.g., only a 0.4% increase for a ResNet-50 mixture of size 10). We perform a systematic empirical study on the choices of prior, variational posterior, and methods to improve training. For ResNet-50 on ImageNet, Wide ResNet 28-10 on CIFAR-10/100, and an RNN on MIMIC-III, rank-1 BNNs achieve state-of-the-art performance across log-likelihood, accuracy, and calibration on the test sets and outof-distribution variants. 1 * Equal contribution † Work completed as a Google AI Resident.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2005.07186")
get_code_for_paper("2005.07186")
have("2005.07186")
The run record, dated, one paper per request, free:
curl https://syntology.ai/api/ran/2005.07186.json
A badge for a README (the split and the date, never a ratio):
[](https://syntology.ai/paper/2005.07186)
Connect an agent — have() is free.