We lifted 2 functions out of this paper's own repositories and ran 0 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| kuntalkumarpal/t5numeracy | canonical | 0 of 2 |
| Function | Status | Where it lives |
|---|---|---|
| compute_exact_match | Not yet run | kuntalkumarpal/t5numeracy/code/decoding.py code served (permissive licence) · get_code("9a405020adf1ed97") |
| value_to_label | Not yet run | kuntalkumarpal/t5numeracy/code/train_magnitude.py code served (permissive licence) · get_code("f8542ff32f2e8f91") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
The transformer-based pre-trained language models have been tremendously successful in most of the conventional NLP tasks. But they often struggle in those tasks where numerical understanding is required. Some possible reasons can be the tokenizers and pre-training objectives which are not specifically designed to learn and preserve numeracy. Here we investigate the ability of text-to-text transfer learning model (T5), which has outperformed its predecessors in the conventional NLP tasks, to learn numeracy. We consider four numeracy tasks: numeration, magnitude order prediction, finding minimum and maximum in a series, and sorting. We find that, although T5 models perform reasonably well in the interpolation setting, they struggle considerably in the extrapolation setting across all four tasks.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2109.04672")
get_code_for_paper("2109.04672")
have("2109.04672")
Connect an agent — have() is free.