We lifted 15 functions out of this paper's own repositories and ran 13 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| the-chen-lab/data-addition-dilemma | canonical | 13 of 15 |
| Function | Status | Where it lives |
|---|---|---|
| flatten | Ran | the-chen-lab/data-addition-dilemma/Yelp-MIMIC/run_dip.py pointer only (licence: NONE) · get_code("8e277d9a0db29745") |
| get_Xy | Ran | the-chen-lab/data-addition-dilemma/Yelp-MIMIC/run_mixture.py pointer only (licence: NONE) · get_code("9efba4f9526a4af4") |
| get_clean_race | Ran | the-chen-lab/data-addition-dilemma/Yelp-MIMIC/run_dip.py pointer only (licence: NONE) · get_code("5893354afbcc35a6") |
| get_data_for_state | Ran | the-chen-lab/data-addition-dilemma/folktables_exp/run_scaling.py pointer only (licence: NONE) · get_code("4ff08a649412b970") |
| get_group | Ran | the-chen-lab/data-addition-dilemma/Yelp-MIMIC/run_dip.py pointer only (licence: NONE) · get_code("55187e06900bef17") |
| get_group | Ran | the-chen-lab/data-addition-dilemma/Yelp-MIMIC/run_sequential.py pointer only (licence: NONE) · get_code("cf9d5d8d97e9a8ff") |
| get_hospital | Ran | the-chen-lab/data-addition-dilemma/kl_utils.py pointer only (licence: NONE) · get_code("3d4a891116d14206") |
| group_accuracy | Ran | the-chen-lab/data-addition-dilemma/folktables_exp/metrics.py pointer only (licence: NONE) · get_code("2235175d7038580c") |
| group_accuracy_ot | Ran | the-chen-lab/data-addition-dilemma/folktables_exp/metrics.py pointer only (licence: NONE) · get_code("b8d7fff533afdcd3") |
| model_choice | Ran | the-chen-lab/data-addition-dilemma/folktables_exp/metrics.py pointer only (licence: NONE) · get_code("fa4bb6e5d361ce50") |
| partition_array | Ran | the-chen-lab/data-addition-dilemma/kl_utils.py pointer only (licence: NONE) · get_code("92f8ecbd9aea7e44") |
| run_experiment | Ran | the-chen-lab/data-addition-dilemma/Yelp-MIMIC/run_mixture.py pointer only (licence: NONE) · get_code("bb6dd2f3e62967b9") |
| run_model | Ran | the-chen-lab/data-addition-dilemma/Yelp-MIMIC/run_mixture.py pointer only (licence: NONE) · get_code("b3ee0f411e006d36") |
| run_yelp_exp_prep | Not yet run | the-chen-lab/data-addition-dilemma/Yelp-MIMIC/run_sequential.py pointer only (licence: NONE) · get_code("797bb0509e425320") |
| yelp_seq_data_prep | Not yet run | the-chen-lab/data-addition-dilemma/Yelp-MIMIC/run_sequential.py pointer only (licence: NONE) · get_code("cfe5cb3002787234") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
In many machine learning for healthcare tasks, standard datasets are constructed by amassing data across many, often fundamentally dissimilar, sources. But when does adding more data help, and when does it hinder progress on desired model outcomes in real-world settings? We identify this situation as the \textit{Data Addition Dilemma}, demonstrating that adding training data in this multi-source scaling context can at times result in reduced overall accuracy, uncertain fairness outcomes, and reduced worst-subgroup performance. We find that this possibly arises from an empirically observed trade-off between model performance improvements due to data scaling and model deterioration from distribution shift. We thus establish baseline strategies for navigating this dilemma, introducing distribution shift heuristics to guide decision-making on which data sources to add in data scaling, in order to yield the expected model performance improvements. We conclude with a discussion of the required considerations for data collection and suggestions for studying data composition and scale in the age of increasingly larger models.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2408.04154")
get_code_for_paper("2408.04154")
have("2408.04154")
Connect an agent — have() is free.