SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2211.16971 · EMNLP · 2022

A Pipeline for Generating, Annotating and Employing Synthetic Data for Real World Question Answering

Maria Liakata, Rob Procter, Matt Maufe, James Ravenscroft

arXiv · PDF · Open in the Atlas

Code that ran

We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

Question Answering (QA) is a growing area of research, often used to facilitate the extraction of information from within documents. Stateof-the-art QA models are usually pre-trained on domain-general corpora like Wikipedia and thus tend to struggle on out-of-domain documents without fine-tuning. We demonstrate that synthetic domain-specific datasets can be generated easily using domain-general models, while still providing significant improvements to QA performance. We present two new tools for this task: A flexible pipeline for validating the synthetic QA data and training downstream models on it, and an online interface to facilitate human annotation of this generated data. Using this interface, crowdworkers labelled 1117 synthetic QA pairs, which we then used to fine-tune downstream models and improve domain-specific QA performance by 8.75 F1.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2211.16971")
get_code_for_paper("2211.16971")
have("2211.16971")

The run record, dated, one paper per request, free:

curl https://syntology.ai/api/ran/2211.16971.json

A badge for a README (the split and the date, never a ratio):

[![Syntology run record](https://syntology.ai/api/ran/2211.16971.svg)](https://syntology.ai/paper/2211.16971)

Connect an agent — have() is free.