Integrating Local and Global Entropy for Uncertainty Quantification in LLMs

Tianyi Zhou, Aristides Gionis, Sanjay Chawla, Johanne Medina, Keivin Isufaj

arXiv · PDF · Open in the Atlas

Code that ran

We lifted 5 functions out of this paper's own repositories and ran 4 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.

Repositories linked to this paper

• qcri/GLU

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

Large language models hallucinate confidently, making uncertainty quantification (UQ) essential for reliable deployment. Existing methods rely predominantly on token-level signals, leaving the geometric structure of intermediate hidden states underused. In this paper, we take the geometric complexity of hidden-state matrices as a measure of the global uncertainty of LLMs, while treating token-level uncertainty estimation as a local metric. We show that hidden-state geometric entropy (global uncertainty) and token-level entropy (local uncertainty) are statistically near-orthogonal, capturing distinct failure regimes for reliability prediction. In particular, global geometry recovers the confident-but-wrong failure mode that local signals systematically miss. Building on this, we propose Global-Local Uncertainty (GLU), an unsupervised, single-pass score that fuses the two signals via a multiplicative gate. Across three model families and six benchmarks, GLU matches or outperforms all unsupervised baselines while requiring only a single forward pass and remaining length-normalized and architecture-agnostic. Code is available on https://github.com/qcri/GLU.git. Hallucination remains one of the most persistent failure modes of large language models (LLMs). Despite rapid advances in capability, frontier systems continue to produce fluent, specific, and incorrect answers. A recent crossdomain benchmark spanning 6,000 questions across 42 topics found that fewer than 1% of evaluated models score above zero on a [-100, 100] reliability index, with the best-performing frontier model reaching only 33 [Jackson et al., 2025]. Uncertainty quantification (UQ) offers a principled path forward, and a wide range of methods have been proposed, from token-level entropy [Zhang et al., 2024] and sampling-based consistency [Farquhar et al., 2024, Yadkori et al., 2024a] to evidential [Sensoy et al., 2018, Ma et al., 2025] and attention-based approaches [Sriramanan et al., 2024, Skean et al., 2025]; yet few are deployed in practice. While other methods impose at least one of the following barriers of multiple generation passes, task-specific supervision, sensitivity to response length, or architectural assumptions, we argue that a practical UQ score should be unsupervised, computed in a single forward pass, length-normalized, and architecture-agnostic.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2606.09875")
get_code_for_paper("2606.09875")
have("2606.09875")

Connect an agent — have() is free.