We lifted 10 functions out of this paper's own repositories and ran 0 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| ekinakyurek/marc | canonical | 0 of 10 |
| Function | Status | Where it lives |
|---|---|---|
| display_messages | Not yet run | ekinakyurek/marc/arclib/messagers.py code served (permissive licence) · get_code("b90ccea3c67381c1") |
| get_all_type_of_votingsv2 | Not yet run | ekinakyurek/marc/arclib/voting.py code served (permissive licence) · get_code("540fdd9083bbe299") |
| np_lru_cache | Not yet run | ekinakyurek/marc/utils/np_cache.py code served (permissive licence) · get_code("14aaa6991d622219") |
| parse_numpy_from_str | Not yet run | ekinakyurek/marc/arclib/representers.py code served (permissive licence) · get_code("96555a8ebf7f060b") |
| read_tasks_from_folder | Not yet run | ekinakyurek/marc/arclib/arc.py code served (permissive licence) · get_code("43bb60cb4cdce0cb") |
| row_base_majority_voting | Not yet run | ekinakyurek/marc/arclib/voting.py code served (permissive licence) · get_code("9700fc1ad6d2c336") |
| setup_lm_api | Not yet run | ekinakyurek/marc/utils/lm_api.py code served (permissive licence) · get_code("942577fea41d2316") |
| to_list | Not yet run | ekinakyurek/marc/arclib/arc.py code served (permissive licence) · get_code("95bd0d71a136300c") |
| to_tuple | Not yet run | ekinakyurek/marc/arclib/arc.py code served (permissive licence) · get_code("77f33bc5b258bb01") |
| vote | Not yet run | ekinakyurek/marc/arclib/voting.py code served (permissive licence) · get_code("9bf200de6989aff4") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Language models (LMs) have shown impressive performance on tasks within their training distribution, but often struggle with structurally novel tasks even when given a small number of in-context task examples. We investigate the effectiveness of test-time training (TTT) -- temporarily updating model parameters during inference using a loss derived from input data -- as a mechanism for improving LMs' reasoning and few-shot learning capabilities. On the Abstraction and Reasoning Corpus (ARC), performing TTT with in-context examples yields up to $6\times$ higher accuracy compared to fine-tuned baselines -- reaching $53.0\%$ on the public validation set with an 8B-parameter LM and $61.9\%$ when ensembled with program-synthesis methods, matching average human performance. On BIG-Bench Hard (BBH), TTT on in-context examples surpasses standard few-shot prompting in the $10$-shot setting by $7.3$ percentage points ($50.5\%$ to $57.8\%$). Our findings highlight the limitations of in-context learning for novel tasks and demonstrate the potential of test-time training to enhance language model adaptability.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2411.07279")
get_code_for_paper("2411.07279")
have("2411.07279")
Connect an agent — have() is free.