SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2102.06571 · ICLR · 2022

Bayesian Neural Network Priors Revisited

Florian Wenzel, Gunnar Rätsch, Vincent Fortuin, Sebastian Ober, Laurence Aitchison, Richard Turner, Mark Van Der Wilk, Adrià Garriga-Alonso

arXiv · PDF · Open in the Atlas

Code that ran

We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

Isotropic Gaussian priors are the de facto standard for modern Bayesian neural network inference. However, it is unclear whether these priors accurately reflect our true beliefs about the weight distributions or give optimal performance. To find better priors, we study summary statistics of neural network weights in networks trained using stochastic gradient descent (SGD). We find that convolutional neural network (CNN) and ResNet weights display strong spatial correlations, while fully connected networks (FCNNs) display heavy-tailed weight distributions. We show that building these observations into priors can lead to improved performance on a variety of image classification datasets. Surprisingly, these priors mitigate the cold posterior effect in FCNNs, but slightly increase the cold posterior effect in ResNets. * Equal contribution. † Equal contribution.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2102.06571")
get_code_for_paper("2102.06571")
have("2102.06571")

The run record, dated, one paper per request, free:

curl https://syntology.ai/api/ran/2102.06571.json

A badge for a README (the split and the date, never a ratio):

[![Syntology run record](https://syntology.ai/api/ran/2102.06571.svg)](https://syntology.ai/paper/2102.06571)

Connect an agent — have() is free.