SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2103.08541 · NAACL · 2021

Get Your Vitamin C! Robust Fact Verification with Contrastive Evidence

Regina Barzilay, Adam Fisch, Tal Schuster

arXiv · PDF · Open in the Atlas

Code that ran

We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

Typical fact verification models use retrieved written evidence to verify claims. Evidence sources, however, often change over time as more information is gathered and revised. In order to adapt, models must be sensitive to subtle differences in supporting evidence. We present VITAMINC, a benchmark infused with challenging cases that require fact verification models to discern and adjust to slight factual changes. We collect over 100,000 Wikipedia revisions that modify an underlying fact, and leverage these revisions, together with additional synthetically constructed ones, to create a total of over 400,000 claim-evidence pairs. Unlike previous resources, the examples in VITAMINC are contrastive, i.e., they contain evidence pairs that are nearly identical in language and content, with the exception that one supports a given claim while the other does not. We show that training using this design increases robustness-improving accuracy by 10% on adversarial fact verification and 6% on adversarial natural language inference (NLI). Moreover, the structure of VITAMINC leads us to define additional tasks for fact-checking resources: tagging relevant words in the evidence for verifying the claim, identifying factual revisions, and providing automatic edits via factually consistent text generation. 1 In animals, spaying involves an invasive removal of the ovaries, but rarely has major complications other than that spayed animals tend to gain weight. In animals, spaying involves an invasive removal of the ovaries, but rarely has major complications; the superstition that it causes weight gain is not based on fact. Claim 1 Spayed animals gain weight. Claim 2 Weight gain in spayed animals is a superstitious myth. Before As of 16 March, more than 182,000 cases of the disease have been reported in over 160 countries and territories, resulting in around 79,000 recoveries and more than 7,100 deaths. After As of 16 March, more than 182,000 cases of the disease have been reported in over 160 countries and territories, resulting in more than 7,100 deaths and around 79,000 recoveries. Before Global hybrid sales are led by the Prius family, with sales of 4.7 million units representing 66.8% of TMC worldwide sales of 7.053 million Lexus and Toyota units through September 2014. After Global hybrid sales are led by the Prius family, with sales of 5.264 million units representing 65.4% of TMC worldwide sales of 8.048 million Lexus and Toyota units delivered through July 2014. Claim 1 Prius sold less than 5 million units, representing over 65.5% of TMC worldwide sales. Claim 2 Prius sold more than 5 million units, representing less than 65.5% of TMC worldwide sales.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2103.08541")
get_code_for_paper("2103.08541")
have("2103.08541")

The run record, dated, one paper per request, free:

curl https://syntology.ai/api/ran/2103.08541.json

A badge for a README (the split and the date, never a ratio):

[![Syntology run record](https://syntology.ai/api/ran/2103.08541.svg)](https://syntology.ai/paper/2103.08541)

Connect an agent — have() is free.