We lifted 1 functions out of this paper's own repositories and ran 1 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| ml-jku/uncertainty-measures | canonical | 1 of 1 |
| Function | Status | Where it lives |
|---|---|---|
| calculate_uncertainties | Ran | ml-jku/uncertainty-measures/source/utils/uncertainty_measures.py code served (permissive licence) · get_code("b893950cdb900d3a") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Reliable estimation of predictive uncertainty is crucial for machine learning applications, particularly in high-stakes scenarios where hedging against risks is essential. Despite its significance, there is no universal agreement on how to best quantify predictive uncertainty. In this work, we revisit core concepts to propose a framework for information-theoretic measures of predictive uncertainty. Our proposed framework categorizes predictive uncertainty measures according to two factors: (I) The predicting model (II) The approximation of the true predictive distribution. Examining all possible combinations of these two factors, we derive a set of predictive uncertainty measures that includes both known and newly introduced ones. We extensively evaluate these measures across a broad set of tasks, identifying conditions under which certain measures excel. Our findings show the importance of aligning the choice of uncertainty measure with the predicting model on in-distribution (ID) data, the limitations of epistemic uncertainty measures for out-of-distribution (OOD) data, and that the disentanglement between measures varies substantially between ID and OOD data. Together, these insights provide a more comprehensive understanding of predictive uncertainty measures, revealing their implicit assumptions and relationships.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2410.10786")
get_code_for_paper("2410.10786")
have("2410.10786")
Connect an agent — have() is free.