Yao Liu, Lihong Li, Chao Zhang, Liang Qiu, Bing Yin, Hyokun Yun, Zhepei Wei, Wenlin Yao, Weizhi Zhang, Qin Lu, Changlong Yu, Puyang Xu
We lifted 8 functions out of this paper's own repositories and ran 7 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| weizhepei/webagent-r1 | canonical | 7 of 8 |
| Function | Status | Where it lives |
|---|---|---|
| action2str | Ran | weizhepei/webagent-r1/WebAgent-R1/Eval/browser_env/actions.py code served (permissive licence) · get_code("482e88612a458c61") |
| get_site_comb_from_filepath | Ran | weizhepei/webagent-r1/WebAgent-R1/Eval/browser_env/auto_login.py code served (permissive licence) · get_code("824ae078c5d3295a") |
| parse_action | Ran | weizhepei/webagent-r1/WebAgent-R1/Eval/browser_env/envs.py code served (permissive licence) · get_code("1829c408a278c974") |
| pil_to_b64 | Ran | weizhepei/webagent-r1/WebAgent-R1/Eval/browser_env/utils.py code served (permissive licence) · get_code("d8254925e6697a59") |
| pil_to_vertex | Ran | weizhepei/webagent-r1/WebAgent-R1/Eval/browser_env/utils.py code served (permissive licence) · get_code("71ec370bd2568e5b") |
| png_bytes_to_numpy | Ran | weizhepei/webagent-r1/WebAgent-R1/Eval/browser_env/utils.py code served (permissive licence) · get_code("04ba3dcf1bb13e4f") |
| remove_unicode | Ran | weizhepei/webagent-r1/WebAgent-R1/Eval/browser_env/processors.py code served (permissive licence) · get_code("b3cfdfe328f76c2d") |
| get_result | Not yet run | weizhepei/webagent-r1/WebAgent-R1/Eval/score.py code served (permissive licence) · get_code("5372a3148cae304b") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
While reinforcement learning (RL) has demonstrated remarkable success in enhancing large language models (LLMs), it has primarily focused on single-turn tasks such as solving math problems. Training effective web agents for multi-turn interactions remains challenging due to the complexity of long-horizon decisionmaking across dynamic web interfaces. In this work, we present WEBAGENT-R1, a simple yet effective end-to-end multi-turn RL framework for training web agents. It learns directly from online interactions with web environments by generating diverse trajectories in parallel, entirely guided by binary rewards depending on task success. Experiments on the WebArena-Lite benchmark demonstrate the effectiveness of WEBAGENT-R1, boosting the task success rate of Qwen-2.5-3B from 6.1% to 33.9% and Llama-3.1-8B from 8.5% to 44.8%, significantly outperforming existing state-of-the-art methods and strong proprietary models such as OpenAI o3. In-depth analyses reveal the effectiveness of the thinking-based prompting strategy and test-time scaling through increased interactions for web tasks. We further investigate different RL initialization policies by introducing two variants, namely WEBAGENT-R1-ZERO and WEBAGENT-R1-COT, which highlight the importance of the warm-up training stage (i.e., behavior cloning) and provide insights on incorporating long chain-of-thought (CoT) reasoning in web agents. 1
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2505.16421")
get_code_for_paper("2505.16421")
have("2505.16421")
Connect an agent — have() is free.