Yangyang Guo, Liqiang Nie, Fangkai Jiao, Yilin Niu, Feng Ji, Feng-Lin Li
We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Pre-trained Language Models (PLMs) have achieved great success on Machine Reading Comprehension (MRC) over the past few years. Although the general language representation learned from large-scale corpora does benefit MRC, the poor support in evidence extraction which requires reasoning across multiple sentences hinders PLMs from further advancing MRC. To bridge the gap between general PLMs and MRC, we present REPT, a REtrieval-based Pre-Training approach. In particular, we introduce two self-supervised tasks to strengthen evidence extraction during pre-training, which is further inherited by downstream MRC tasks through the consistent retrieval operation and model architecture. To evaluate our proposed method, we conduct extensive experiments on five MRC datasets that require collecting evidence from and reasoning across multiple sentences. Experimental results demonstrate the effectiveness of our pre-training approach. Moreover, further analysis shows that our approach is able to enhance the capacity of evidence extraction without explicit supervision.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2105.04201")
get_code_for_paper("2105.04201")
have("2105.04201")
The run record, dated, one paper per request, free:
curl https://syntology.ai/api/ran/2105.04201.json
A badge for a README (the split and the date, never a ratio):
[](https://syntology.ai/paper/2105.04201)
Connect an agent — have() is free.