SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2502.14409 · EMNLP · 2025

Unstructured Evidence Attribution for Long Context Query Focused Summarization

Lu Wang, David Jurgens, Dustin Wright, Isabelle Augenstein, Zain Muhammad Mujahid

arXiv · PDF · Open in the Atlas

Code that ran

We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

Large language models (LLMs) are capable of generating coherent summaries from very long contexts given a user query, and extracting and citing evidence spans helps improve the trustworthiness of these summaries. Whereas previous work has focused on evidence citation with fixed levels of granularity (e.g. sentence, paragraph, document, etc.), we propose to extract unstructured (i.e., spans of any length) evidence in order to acquire more relevant and consistent evidence than in the fixed granularity case. We show how existing systems struggle to copy and properly cite unstructured evidence, which also tends to be "lost-in-the-middle". To help models perform this task, we create the Summaries with Unstructured Evidence Text dataset (SUnsET), a synthetic dataset generated using a novel pipeline, which can be used as training supervision for unstructured evidence summarization. We demonstrate across 5 LLMs and 4 datasets spanning human written, synthetic, single, and multi-document settings that LLMs adapted with SUnsET generate more relevant and factually consistent evidence with their summaries, extract evidence from more diverse locations in their context, and can generate more relevant and consistent summaries than baselines with no fine-tuning and fixed granularity evidence. We release SUnsET and our generation code to the public. 1 Fixed-Granular Single Sentence Citation: SUMMARY SNIPPET: ... [48] explains that the legend of the Ghost Ship is often told by space men as a cautionary tale.... EVIDENCE: [48] He had heard it spoken of in whispers by drunken space men and professional tellers of fairy tales. Unstructured Citation: SUMMARY SNIPPET: ...he, like the ship's former crew, is doomed to wander in space, never able to return to Earth, a haunting reminder of what he has lost and what he can never have [2]...

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2502.14409")
get_code_for_paper("2502.14409")
have("2502.14409")

The run record, dated, one paper per request, free:

curl https://syntology.ai/api/ran/2502.14409.json

A badge for a README (the split and the date, never a ratio):

[![Syntology run record](https://syntology.ai/api/ran/2502.14409.svg)](https://syntology.ai/paper/2502.14409)

Connect an agent — have() is free.