Nuria Oliver, Sepp Hochreiter, Kajetan Schweighofer, Adrian Arnaiz-Rodriguez
We lifted 6 functions out of this paper's own repositories and ran 5 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| ml-jku/disparate-benefits | canonical | 5 of 6 |
| Function | Status | Where it lives |
|---|---|---|
| accuracy | Ran | ml-jku/disparate-benefits/source/utils/metrics.py code served (permissive licence) · get_code("b1493aff33edf7b8") |
| auroc | Ran | ml-jku/disparate-benefits/source/utils/metrics.py code served (permissive licence) · get_code("442722aa37fb0ef0") |
| balanced_accuracy | Ran | ml-jku/disparate-benefits/source/utils/metrics.py code served (permissive licence) · get_code("720ea3efc589e9b3") |
| evaluate | Ran | ml-jku/disparate-benefits/source/utils/train_utils.py code served (permissive licence) · get_code("7262c1188d1b942a") |
| update | Ran | ml-jku/disparate-benefits/source/utils/train_utils.py code served (permissive licence) · get_code("9c40d52fe7476ce1") |
| fit | Not yet run | ml-jku/disparate-benefits/source/utils/train_utils.py code served (permissive licence) · get_code("dd461a45f2667e8c") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Ensembles of Deep Neural Networks, Deep Ensembles, are widely used as a simple way to boost predictive performance. However, their impact on algorithmic fairness is not well understood yet. Algorithmic fairness examines how a model's performance varies across socially relevant groups defined by protected attributes such as age, gender, or race. In this work, we explore the interplay between the performance gains from Deep Ensembles and fairness. Our analysis reveals that they unevenly favor different groups, a phenomenon that we term the disparate benefits effect. We empirically investigate this effect using popular facial analysis and medical imaging datasets with protected group attributes and find that it affects multiple established group fairness metrics, including statistical parity and equal opportunity. Furthermore, we identify that the per-group differences in predictive diversity of ensemble members can explain this effect. Finally, we demonstrate that the classical Hardt post-processing method is particularly effective at mitigating the disparate benefits effect of Deep Ensembles by leveraging their better-calibrated predictive distributions.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2410.13831")
get_code_for_paper("2410.13831")
have("2410.13831")
Connect an agent — have() is free.