SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2502.11705 · ACL · 2025

LLM Agents Making Agent Tools

Dyke Ferber, G. Wölflein, D. Truhn, Ognjen Arandjelovic, J. Kather

arXiv · PDF · Open in the Atlas

Code that ran

We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

Tool use has turned large language models (LLMs) into powerful agents that can perform complex multi-step tasks by dynamically utilising external software components. However, these tools must be implemented in advance by human developers, hindering the applicability of LLM agents in domains demanding large numbers of highly specialised tools, like in life sciences and medicine. Motivated by the growing trend of scientific studies accompanied by public code repositories, we propose TOOLMAKER, an agentic framework that autonomously transforms papers with code into LLM-compatible tools. Given a GitHub URL and short task description, TOOLMAKER autonomously installs dependencies and generates code to perform the task, using a closedloop self-correction mechanism for debugging. To evaluate our approach, we introduce a benchmark comprising 15 complex computational tasks spanning various domains with over 100 unit tests to assess correctness and robustness. Our method correctly implements 80% of the tasks, substantially outperforming current state-of-the-art software engineering agents. TOOLMAKER therefore is a step towards fully autonomous agent-based scientific workflows 1 .

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2502.11705")
get_code_for_paper("2502.11705")
have("2502.11705")

The run record, dated, one paper per request, free:

curl https://syntology.ai/api/ran/2502.11705.json

A badge for a README (the split and the date, never a ratio):

[![Syntology run record](https://syntology.ai/api/ran/2502.11705.svg)](https://syntology.ai/paper/2502.11705)

Connect an agent — have() is free.