Test-Time Augmentation for LLMs: When Input Diversity Beats Output Diversity at Matched Compute

Nikita Kozodoi, Zainab Afolabi, Jack Butler

arXiv · PDF · Open in the Atlas

Code that ran

We lifted 8 functions out of this paper's own repositories and ran 7 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.

Repositories linked to this paper

• aws-samples/sample-genai-reflection-for-bedrock

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

Test-time scaling improves LLM accuracy but multiplies inference cost, making the accuracy gained per unit of compute the metric that matters in deployment. Self-consistency is one of the established approaches, which spends this budget entirely on the output side by sampling repeated reasoning paths. We study Test-Time Augmentation (TTA), which extends self-consistency by also perturbing the input, aggregating predictions across transformed versions of the input, and ask whether input-side diversity converts compute into accuracy more efficiently than outputside diversity. We perform a systematic, matched-compute comparison: we evaluate three simple input-side strategies (semantic rephrasing, lexical perturbations, and visual transformations) across six datasets covering general and multilingual knowledge, mathematical reasoning, multimodal question answering, and sentiment classification, against chainof-thought prompting and self-consistency. Semantic rephrasing delivers consistent and statistically significant accuracy gains while Paretodominating self-consistency on cost-effectiveness, delivering roughly 1.8× more accuracy per dollar and outperforming it on five of six tasks. We further analyze the number of augmentations, multi-modal strategies, and base model scaling, finding that TTA is most cost-effective for mid-tier models where a stronger model is unavailable or too expensive. Our findings indicate that for current mid-tier LLMs, varying the input converts inference compute into accuracy more efficiently than varying the reasoning path alone. The TTA implementation is available at https: //github.com/aws-samples/sample-genai-reflection-for-bedrock.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2608.09351")
get_code_for_paper("2608.09351")
have("2608.09351")

Connect an agent — have() is free.