We lifted 4 functions out of this paper's own repositories and ran 4 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| pprp/acbench | canonical | 2 of 2 |
| copy not recorded | — | 2 of 2 |
| Function | Status | Where it lives |
|---|---|---|
| calculate_energy | Ran | pprp/acbench/acbench/analysis/analyze_energy/run_energy_analysis.py code served (permissive licence) · get_code("fa1ded26143f04d3") |
| process_batch | Ran | pprp/acbench/acbench/analysis/analyze_energy/run_energy_analysis.py code served (permissive licence) · get_code("f1f3b4b8aaac5ce6") |
| workflow_to_graph_list | Ran | this paper's copy was not recorded; identical code first harvested from zjunlp/worfbench pointer only · get_code("8e2012e57dc7319f") |
| workflow_to_node_list | Ran | this paper's copy was not recorded; identical code first harvested from zjunlp/worfbench pointer only · get_code("c8a83aad8c019dff") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Post-training compression reduces the computational and memory costs of large language models (LLMs), enabling resource-efficient deployment. However, existing compression benchmarks only focus on language modeling (e.g., perplexity) and natural language understanding tasks (e.g., GLUE accuracy), ignoring the agentic capabilities - workflow, tool use/function call, long-context understanding and real-world application. We introduce the Agent Compression Benchmark (ACBench), the first comprehensive benchmark for evaluating how compression impacts LLMs' agentic abilities. ACBench spans (1) 12 tasks across 4 capabilities (e.g., WorfBench for workflow generation, Needle-in-Haystack for long-context retrieval), (2) quantization (GPTQ, AWQ) and pruning (Wanda, SparseGPT), and (3) 15 models, including small (Gemma-2B), standard (Qwen2.5 7B-32B), and distilled reasoning LLMs (DeepSeek-R1-Distill). Our experiments reveal compression tradeoffs: 4-bit quantization preserves workflow generation and tool use (1%-3% drop) but degrades real-world application accuracy by 10%-15%. We introduce ERank, Top-k Ranking Correlation and Energy to systematize analysis. ACBench provides actionable insights for optimizing LLM compression in agentic scenarios. The code can be found in https://github.com/pprp/ACBench.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2505.19433")
get_code_for_paper("2505.19433")
have("2505.19433")
Connect an agent — have() is free.