Matias Valdenegro-Toro
We lifted 3 functions out of this paper's own repositories and ran 3 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| mvaldenegro/paper-quality-epistemic-uncertainty-bayes | canonical | 3 of 3 |
| Function | Status | Where it lives |
|---|---|---|
| class_sampling | Ran | mvaldenegro/paper-quality-epistemic-uncertainty-bayes/code/evaluate_grad_uncertainty_samplecomp_entropy.py pointer only (licence: GPL-3.0) · get_code("3b69f2c758e6e837") |
| lower_whisker | Ran | mvaldenegro/paper-quality-epistemic-uncertainty-bayes/code/evaluate_grad_uncertainty_samplecomp_entropy.py pointer only (licence: GPL-3.0) · get_code("9a370d64d980712f") |
| upper_whisker | Ran | mvaldenegro/paper-quality-epistemic-uncertainty-bayes/code/evaluate_grad_uncertainty_samplecomp_entropy.py pointer only (licence: GPL-3.0) · get_code("0a72aaf87bf88297") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Uncertainty quantification in neural network promises to increase safety of AI systems, but it is not clear how performance might vary with the training set size. In this paper we evaluate seven uncertainty methods on Fashion MNIST and CIFAR10, as we sub-sample and produce varied training set sizes. We find that calibration error and out of distribution detection performance strongly depend on the training set size, with most methods being miscalibrated on the test set with small training sets. Gradient-based methods seem to poorly estimate epistemic uncertainty and are the most affected by training set size. We expect our results can guide future research into uncertainty quantification and help practitioners select methods based on their particular available data.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2111.09808")
get_code_for_paper("2111.09808")
have("2111.09808")
Connect an agent — have() is free.