James Thorne, Jiwoo Hong, Guijin Son, Hyunwoo Ko
We lifted 6 functions out of this paper's own repositories and ran 2 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| gauss5930/mclm | — | 2 of 6 |
| Function | Status | Where it lives |
|---|---|---|
| format_prompt | Ran | gauss5930/mclm/src/prm.py pointer only (licence: NONE) · get_code("b960d1f9e15f6bd3") |
| score_candidate | Ran | gauss5930/mclm/src/prm.py pointer only (licence: NONE) · get_code("09ee312c8e189386") |
| batch_completion | Not yet run | gauss5930/mclm/src/prm.py pointer only (licence: NONE) · get_code("ae97665c18d73ffe") |
| iterative_prm | Not yet run | gauss5930/mclm/src/prm.py pointer only (licence: NONE) · get_code("79004b97a648cc42") |
| log_info | Not yet run | gauss5930/mclm/src/prm.py pointer only (licence: NONE) · get_code("e44973ad91efe004") |
| single_completion | Not yet run | gauss5930/mclm/src/prm.py pointer only (licence: NONE) · get_code("fed1bf9a40096896") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Scaling pre-training compute has proven effective for achieving multilinguality, but does the same hold for test-time scaling? In this work, we introduce MCLM, a multilingual math benchmark featuring competition-level problems in 55 languages. We test three test-time scaling methods-Outcome Reward Modeling (ORM), Process Reward Modeling (PRM), and Budget Forcing (BF)-on both Qwen2.5-1.5B Math and MR1-1.5B, a multilingual LLM we trained for extended reasoning. Our experiments show that using Qwen2.5-1.5B Math with ORM achieves a score of 35.8 on MCLM, while BF on MR1-1.5B attains 35.2. Although "thinking LLMs" have recently garnered significant attention, we find that their performance is comparable to traditional scaling methods like best-of-N once constrained to similar levels of inference FLOPs. Moreover, while BF yields a 20-point improvement on English AIME, it provides only a 1.94-point average gain across other languages-a pattern consistent across the other test-time scaling methods we studied-highlighting that test-time scaling may not generalize as effectively to multilingual tasks. To foster further research, we release MCLM, MR1-1.5B, and evaluation results. 1
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2502.17407")
get_code_for_paper("2502.17407")
have("2502.17407")
Connect an agent — have() is free.