Zhangxuan Gu, Changhua Meng, Sunhao Dai, Yuqi Zhou, Zhuohan Cai, Shuheng Shen, Yichen Gong
We lifted 4 functions out of this paper's own repositories and ran 4 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| inclusionAI/UI-Venus | canonical | 4 of 4 |
| Function | Status | Where it lives |
|---|---|---|
| history_reply | Ran | inclusionAI/UI-Venus/models/browser/venus_browser.py pointer only (licence: NONE) · get_code("3fff0230e12ea460") |
| parse_action | Ran | inclusionAI/UI-Venus/models/browser/venus_browser.py pointer only (licence: NONE) · get_code("df341ba22d2a7a95") |
| run_adb_cmd | Ran | inclusionAI/UI-Venus/Venus_framework/Venus_framework_mobile/app/verify.py pointer only (licence: NONE) · get_code("f2f535232e7d0b1c") |
| setup_logging | Ran | inclusionAI/UI-Venus/Venus_framework/Venus_framework_mobile/batch_runner.py pointer only (licence: NONE) · get_code("8c4dadfe748db74a") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Existing online benchmarks for mobile GUI agents remain largely app-centric and taskhomogeneous, failing to reflect the diversity and instability of real-world mobile usage. To this end, we introduce VenusBench-Mobile, a challenging online benchmark for evaluating general-purpose mobile GUI agents under realistic, user-centric conditions. VenusBench-Mobile builds two core evaluation pillars: defining what to evaluate via user-intent-driven task design that reflects real mobile usage, and how to evaluate through a capability-oriented annotation scheme for fine-grained agent behavior analysis. Extensive evaluation of stateof-the-art mobile GUI agents reveals large performance gaps relative to prior benchmarks, indicating that VenusBench-Mobile poses substantially more challenging and realistic tasks and that current agents remain far from reliable real-world deployment. Diagnostic analysis further shows that failures are dominated by deficiencies in perception and memory, which are largely obscured by coarse-grained evaluations. Moreover, even the strongest agents exhibit near-zero success under environment variations, highlighting their brittleness in realistic settings. Based on these insights, we believe VenusBench-Mobile provides an important stepping stone toward robust real-world deployment of mobile GUI agents. Code and data are available at https://github.com/inclusionAI/UI-Venus/tree/VenusBench-Mobile.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2604.06182")
get_code_for_paper("2604.06182")
have("2604.06182")
Connect an agent — have() is free.