Huaxiu Yao, Chelsea Finn, Wei Pang, Koh, Yoonho Lee, Bochuan Cao, Caroline Choi
We lifted 2 functions out of this paper's own repositories and ran 0 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| huaxiuyao/wild-time | canonical | 0 of 2 |
| Function | Status | Where it lives |
|---|---|---|
| init | Not yet run | huaxiuyao/wild-time/wildtime/baseline_trainer.py code served (permissive licence) · get_code("beca3cd43469c27a") |
| trainer_init | Not yet run | huaxiuyao/wild-time/wildtime/baseline_trainer.py code served (permissive licence) · get_code("60f3099a9cae2b5e") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Distribution shift occurs when the test distribution differs from the training distribution, and it can considerably degrade performance of machine learning models deployed in the real world. Temporal shifts -distribution shifts arising from the passage of time -often occur gradually and have the additional structure of timestamp metadata. By leveraging timestamp metadata, models can potentially learn from trends in past distribution shifts and extrapolate into the future. While recent works have studied distribution shifts, temporal shifts remain underexplored. To address this gap, we curate Wild-Time, a benchmark of 5 datasets that reflect temporal distribution shifts arising in a variety of real-world applications, including patient prognosis and news classification. On these datasets, we systematically benchmark 13 prior approaches, including methods in domain generalization, continual learning, self-supervised learning, and ensemble learning. We use two evaluation strategies: evaluation with a fixed time split (Eval-Fix) and evaluation with a data stream (Eval-Stream). Eval-Fix, our primary evaluation strategy, aims to provide a simple evaluation protocol, while Eval-Stream is more realistic for certain real-world applications. Under both evaluation strategies, we observe an average performance drop of 20% from in-distribution to out-of-distribution data. Existing methods are unable to close this gap. Code is available at https://wildtime.github.io/.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2211.14238")
get_code_for_paper("2211.14238")
have("2211.14238")
Connect an agent — have() is free.