Naren Ramakrishnan, Mandar Sharma, Nikhil Muralidhar
We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
The field of Math-NLP has witnessed significant growth in recent years, motivated by the desire to expand LLM performance to the learning of non-linguistic notions (numerals, and subsequently, arithmetic reasoning). However, non-linguistic skill injection typically comes at a cost for LLMs: it leads to catastrophic forgetting of core linguistic skills, a consequence that often remains unaddressed in the literature. As Math-NLP has been able to create LLMs that can closely approximate the mathematical skills of a grade-schooler or the arithmetic reasoning skills of a calculator, the practicality of these models fail if they concomitantly shed their linguistic capabilities. In this work, we take a closer look into the phenomena of catastrophic forgetting as it pertains to LLMs and subsequently offer a novel framework for non-linguistic skill injection for LLMs based on information-theoretic interventions and skill-specific losses that enable the learning of strict arithmetic reasoning. Our model outperforms the state-of-the-art both on injected non-linguistic skills and on linguistic knowledge retention, and does so with a fraction of the non-linguistic training data (1/4) and zero additional synthetic linguistic training data. Our pre-trained models and experimentation codebases are hosted online 1 .
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2305.08246")
get_code_for_paper("2305.08246")
have("2305.08246")
The run record, dated, one paper per request, free:
curl https://syntology.ai/api/ran/2305.08246.json
A badge for a README (the split and the date, never a ratio):
[](https://syntology.ai/paper/2305.08246)
Connect an agent — have() is free.