Somak Aditya, Debrup Das, Debopriyo Banerjee, Ashish Kulkarni
We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Tool-augmented Large Language Models (TALMs) are known to enhance the skillset of large language models (LLMs), thereby, leading to their improved reasoning abilities across many tasks. While, TALMs have been successfully employed in different questionanswering benchmarks, their efficacy on complex mathematical reasoning benchmarks, and the potential complementary benefits offered by tools for knowledge retrieval and mathematical equation solving are open research questions. In this work, we present MATHSEN-SEI, a tool-augmented large language model for mathematical reasoning. We study the complementary benefits of the tools -knowledge retriever (Bing Web Search), program generator + executor (Python), and symbolic equation solver (WolframAlpha-API) through evaluations on mathematical reasoning datasets. We perform exhaustive ablations on MATH, a popular dataset for evaluating mathematical reasoning on diverse mathematical disciplines. We also conduct experiments involving well-known tool planners to study the impact of tool sequencing on the model performance. MATHSENSEI achieves 13.5% better accuracy over gpt-3.5-turbo with Chain-of-Thought on the MATH dataset. We further observe that TALMs are not as effective for simpler math word problems (in GSM-8K), and the benefit increases as the complexity and required knowledge increases (progressively over AQuA, MMLU-Math, and higher level complex questions in MATH). The code and data are available at https://github.com/Debrup-61/MathSensei * Work done during internship at Rakuten India Enterprise Private Limited.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2402.17231")
get_code_for_paper("2402.17231")
have("2402.17231")
The run record, dated, one paper per request, free:
curl https://syntology.ai/api/ran/2402.17231.json
A badge for a README (the split and the date, never a ratio):
[](https://syntology.ai/paper/2402.17231)
Connect an agent — have() is free.