SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2412.21139 · ICML · 2025

Training Software Engineering Agents and Verifiers with SWE-Gym

Graham Neubig, Yizhe Zhang, Heng Ji, Navdeep Jaitly, Alane Suhr, Xingyao Wang, Jiayi Pan

arXiv · PDF · Open in the Atlas

Code that ran

We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

We present SWE-Gym, the first environment for training software engineering (SWE) agents. SWE-Gym contains 2,438 real-world task instances, each comprising a Python codebase with an executable runtime environment, unit tests, and a task specified in natural language. We use SWE-Gym to train language model based SWE agents, and achieve up to 19% absolute gains in resolution rate on the popular SWE-Bench Verified and Lite test sets. We also experiment with inferencetime scaling through verifiers trained on agent trajectories sampled from SWE-Gym. When combined with our fine-tuned SWE agents, we achieve 32.0% and 26.0% on SWE-Bench Verified and Lite, respectively, reflecting a new state-of-the-art for open-weight SWE agents. To facilitate further research, we publicly release SWE-Gym, models, and agent trajectories. * Equal contribution. † Equal supervision. 1 UC Berkeley 2 UIUC 3 CMU 4 Apple.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2412.21139")
get_code_for_paper("2412.21139")
have("2412.21139")

The run record, dated, one paper per request, free:

curl https://syntology.ai/api/ran/2412.21139.json

A badge for a README (the split and the date, never a ratio):

[![Syntology run record](https://syntology.ai/api/ran/2412.21139.svg)](https://syntology.ai/paper/2412.21139)

Connect an agent — have() is free.