Michael Hanna, Ollie Liu, Alexandre Variengien
We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Pre-trained language models can be surprisingly adept at tasks they were not explicitly trained on, but how they implement these capabilities is poorly understood. In this paper, we investigate the basic mathematical abilities often acquired by pre-trained language models. Concretely, we use mechanistic interpretability techniques to explain the (limited) mathematical abilities of GPT-2 small. As a case study, we examine its ability to take in sentences such as "The war lasted from the year 1732 to the year 17", and predict valid two-digit end years (years > 32). We first identify a circuit, a small subset of GPT-2 small's computational graph that computes this task's output. Then, we explain the role of each circuit component, showing that GPT-2 small's final multi-layer perceptrons boost the probability of end years greater than the start year. Finally, we find related tasks that activate our circuit. Our results suggest that GPT-2 small computes greater-than using a complex mechanism that activates across diverse contexts. * Work performed as part of Redwood Research's REMIX program. † Work performed during an internship. Now at Conjecture. 37th Conference on Neural Information Processing Systems (NeurIPS 2023).
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2305.00586")
get_code_for_paper("2305.00586")
have("2305.00586")
The run record, dated, one paper per request, free:
curl https://syntology.ai/api/ran/2305.00586.json
A badge for a README (the split and the date, never a ratio):
[](https://syntology.ai/paper/2305.00586)
Connect an agent — have() is free.