Yijun Yang, Víctor Gutiérrez-Basulto, Jeff Pan, Jie He, Pinzhen Chen
We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Several recent papers have investigated the potential of language models as knowledge bases as well as the existence of severe biases when extracting factual knowledge. In this work, we focus on the factual probing performance over unseen prompts from tuning, and using a probabilistic view we show the inherent misalignment between pre-training and downstream tuning objectives in language models for probing knowledge. We hypothesize that simultaneously debiasing these objectives can be the key to generalisation over unseen prompts. We propose an adapter-based framework, UniArk, for generalised and consistent factual knowledge extraction through simple methods without introducing extra parameters. Extensive experiments show that UniArk can significantly improve the model's out-of-domain generalisation as well as consistency under various prompts. Additionally, we construct ParaTrex, a largescale and diverse dataset for measuring the inconsistency and out-of-domain generation of models. Further, ParaTrex offers a reference method for constructing paraphrased datasets using large language models. 1 English Spanish French Extract (s, OfficialLanguage, o) by prompt � � Factual Knowledge Extraction P(object | subject, template) The official language of Sorengo is [mask]. The official language of [mask] is [mask]. UniArk English Spanish French Bias from P(object | template), P(subject | template) Template Prior Bias 1. [mask] is the language officially designated by c. 2. Sorengo recognizes [mask] as its official language.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2404.01253")
get_code_for_paper("2404.01253")
have("2404.01253")
The run record, dated, one paper per request, free:
curl https://syntology.ai/api/ran/2404.01253.json
A badge for a README (the split and the date, never a ratio):
[](https://syntology.ai/paper/2404.01253)
Connect an agent — have() is free.