Michael Schlichtkrull, Zhijiang Guo, Andreas Vlachos
We lifted 1 functions out of this paper's own repositories and ran 1 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| MichSchli/AVeriTeC | canonical | 1 of 1 |
| Function | Status | Where it lives |
|---|---|---|
| shift_tokens_right | Ran | MichSchli/AVeriTeC/retrieval_reranking/decorate_with_questions.py pointer only (licence: NONE) · get_code("5a0796818cb2aaa5") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Existing datasets for automated fact-checking have substantial limitations, such as relying on artificial claims, lacking annotations for evidence and intermediate reasoning, or including evidence published after the claim. In this paper, we introduce AVERITEC, a new dataset of 4,568 real-world claims covering factchecks by 50 different organizations. Each claim is annotated with questionanswer pairs supported by evidence available online, as well as textual justifications explaining how the evidence combines to produce a verdict. Through a multiround annotation process, we avoid common pitfalls including context dependence, evidence insufficiency, and temporal leakage, and reach a substantial inter-annotator agreement of κ = 0.619 on verdicts. We develop a baseline as well as an evaluation scheme for verifying claims through question-answering against the open web.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2305.13117")
get_code_for_paper("2305.13117")
have("2305.13117")
Connect an agent — have() is free.