SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2401.16475 · ACL · 2024

INFOLOSSQA: Characterizing and Recovering Information Loss in Text Simplification

Junyi Li, Wei Xu, Kyle Lo, Byron Wallace, Jörg Schlötterer, Christin Seifert, Sebastian Joseph, Jan Trienes

arXiv · PDF · Open in the Atlas

Code that ran

We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

Text simplification aims to make technical texts more accessible to laypeople but often results in deletion of information and vagueness. This work proposes INFOLOSSQA, a framework to characterize and recover simplificationinduced information loss in form of questionand-answer (QA) pairs. Building on the theory of Questions Under Discussion, the QA pairs are designed to help readers deepen their knowledge of a text. First, we collect a dataset of 1,000 linguist-curated QA pairs derived from 104 LLM simplifications of English medical study abstracts. Our analyses of this data reveal that information loss occurs frequently, and that the QA pairs give a high-level overview of what information was lost. Second, we devise two methods for this task: end-to-end prompting of open-source and commercial language models, and a natural language inference pipeline. With a novel evaluation framework considering the correctness of QA pairs and their linguistic suitability, our expert evaluation reveals that models struggle to reliably identify information loss and applying similar standards as humans at what constitutes information loss. 1 * Work done while visiting UT Austin. 1 Code, dataset and an interactive data viewer is available at https://jantrienes.github.io/ts-info-loss/.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2401.16475")
get_code_for_paper("2401.16475")
have("2401.16475")

The run record, dated, one paper per request, free:

curl https://syntology.ai/api/ran/2401.16475.json

A badge for a README (the split and the date, never a ratio):

[![Syntology run record](https://syntology.ai/api/ran/2401.16475.svg)](https://syntology.ai/paper/2401.16475)

Connect an agent — have() is free.