SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2504.12991 · 2025

A Theoretical Framework for OOD Robustness in Transformers using Gevrey Classes

arXiv · PDF · Open in the Atlas

Code that ran

We lifted 3 functions out of this paper's own repositories and ran 2 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.

RepositoryRoleRan
d09942015ntu/cot_ood_latent canonical 2 of 3
FunctionStatusWhere it lives
calc_kp_divergence Ran d09942015ntu/cot_ood_latent/run_generate_dataset.py
pointer only (licence: NONE) · get_code("2ba42f30673bdd5b")
calc_symmmetric_kl Ran d09942015ntu/cot_ood_latent/run_generate_dataset.py
pointer only (licence: NONE) · get_code("245ecdff1fe2f7ce")
generate_sample Not yet run d09942015ntu/cot_ood_latent/run_generate_dataset.py
pointer only (licence: NONE) · get_code("dac5779ed8dec653")

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

We study the robustness of Transformer language models under semantic out-of-distribution (OOD) shifts, where training and test data lie in disjoint latent spaces. Using Wasserstein-1 distance and Gevrey-class smoothness, we derive sub-exponential upper bounds on prediction error. Our theoretical framework explains how smoothness governs generalization under distributional drift. We validate these findings through controlled experiments on arithmetic and Chain-of-Thought tasks with latent permutations and scalings. Results show empirical degradation aligns with our bounds, highlighting the geometric and functional principles underlying OOD generalization in Transformers.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2504.12991")
get_code_for_paper("2504.12991")
have("2504.12991")

Connect an agent — have() is free.