Luca Scharr, Vanessa Toborek
We lifted 3 functions out of this paper's own repositories and ran 3 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| luca-scharr/cupcurriculum | canonical | 3 of 3 |
| Function | Status | Where it lives |
|---|---|---|
| generate_square_subsequent_mask | Ran | luca-scharr/cupcurriculum/transformer_modell.py code served (permissive licence) · get_code("c7b84365cf3df5a0") |
| hodges_lehmann_estimator | Ran | luca-scharr/cupcurriculum/plot_generator.py code served (permissive licence) · get_code("410038cda15750c9") |
| print_nonzeros | Ran | luca-scharr/cupcurriculum/utils.py code served (permissive licence) · get_code("6aeee88cf4630d23") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Curriculum learning (CL) aims to increase the performance of a learner on a given task by applying a specialized learning strategy. This strategy focuses on either the dataset, the task, or the model. There is little to no work analysing the possibilities to apply CL on the model capacity in natural language processing. To close this gap, we propose the cup curriculum. In a first phase of training we use a variation of iterative magnitude pruning to reduce model capacity. These weights are reintroduced in a second phase, resulting in the model capacity to show a cup-shaped curve over the training iterations. We empirically evaluate different strategies of the cup curriculum and show that it outperforms early stopping reliably while exhibiting a high resilience to overfitting.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2311.03956")
get_code_for_paper("2311.03956")
have("2311.03956")
Connect an agent — have() is free.