We lifted 10 functions out of this paper's own repositories and ran 4 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| rucaibox/chainlm | canonical | 4 of 10 |
| Function | Status | Where it lives |
|---|---|---|
| get_input | Ran | rucaibox/chainlm/debate/debate4cot.py pointer only (licence: NONE) · get_code("9fecbd5be33d8972") |
| get_res_batch | Ran | rucaibox/chainlm/generate/complicate_filter.py pointer only (licence: NONE) · get_code("ee1261cc031b3ee4") |
| get_res_batch | Ran | rucaibox/chainlm/generate/generate_cot.py pointer only (licence: NONE) · get_code("41f2fda3501db450") |
| get_res_batch | Ran | rucaibox/chainlm/generate/specify_filter.py pointer only (licence: NONE) · get_code("5ddeec251ac6468c") |
| annotate | Not yet run | rucaibox/chainlm/generate/filter_claude.py pointer only (licence: NONE) · get_code("2cf2868f64341a27") |
| get_res_batch | Not yet run | rucaibox/chainlm/generate/diversify_filter.py pointer only (licence: NONE) · get_code("12f7e3d5087fec14") |
| get_res_batch | Not yet run | rucaibox/chainlm/generate/filter_chatgpt.py pointer only (licence: NONE) · get_code("cce269798b70d493") |
| get_res_batch | Not yet run | rucaibox/chainlm/generate/generate_question.py pointer only (licence: NONE) · get_code("3ac1b552c4ac7840") |
| get_res_batch | Not yet run | rucaibox/chainlm/generate/rewrite_cot.py pointer only (licence: NONE) · get_code("0d18ca984c151622") |
| load_model | Not yet run | rucaibox/chainlm/debate/debate4cot.py pointer only (licence: NONE) · get_code("5872a9172aa4cf7b") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Chain-of-Thought (CoT) prompting can enhance the reasoning capabilities of large language models (LLMs), establishing itself as a primary approach to solving complex reasoning tasks. Existing CoT synthesis approaches usually focus on simpler reasoning tasks and thus result in low-quality and inconsistent CoT prompts. In response to this challenge, we present an empirical investigation of CoT prompting and introduce CoTGenius, a novel framework designed for the automatic generation of superior CoT prompts. CoTGenius is developed based on three major evolution strategies, i.e., complicate, diversify, and specify-alongside two filtering mechanisms: evolutionary success judgement and correctness verification. We further employ CoTGenius to create an extensive CoT dataset, and subsequently fine-tune the Llama 2-Chat 7B and 13B models on this dataset. We call the resulting model ChainLM. To deal with the cumulative error issue in reasoning steps, we propose a step-level debating method, wherein multiple debaters discuss each reasoning step to arrive at the correct answer. Extensive experiments demonstrate that our ChainLM models exhibit enhanced proficiency in addressing a spectrum of complex reasoning problems compared to existing models. In addition, we conduct an in-depth analysis of the impact of data categories within CoTGenius on the model performance. We release our dataset and code at https://github.com/RUCAIBox/ChainLM.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2403.14312")
get_code_for_paper("2403.14312")
have("2403.14312")
Connect an agent — have() is free.