We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Large language models (LLMs) have exhibited impressive competence in various tasks, but their internal mechanisms on mathematical problems are still under-explored. In this paper, we study a fundamental question: how language models encode the value of numbers, a basic element in math. To study the question, we construct a synthetic dataset comprising addition problems and utilize linear probes to read out input numbers from the hidden states. Experimental results support the existence of encoded number values in LLMs on different layers, and these values can be extracted via linear probes. Further experiments show that LLMs store their calculation results in a similar manner, and we can intervene the output via simple vector additions, proving the causal connection between encoded numbers and language model outputs. Our research provides evidence that LLMs encode the value of numbers linearly, offering insights for better exploring, designing, and utilizing numeric information in LLMs.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2401.03735")
get_code_for_paper("2401.03735")
have("2401.03735")
The run record, dated, one paper per request, free:
curl https://syntology.ai/api/ran/2401.03735.json
A badge for a README (the split and the date, never a ratio):
[](https://syntology.ai/paper/2401.03735)
Connect an agent — have() is free.