We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
The recent success of large pre-trained language models such as BERT and GPT-2 has suggested the effectiveness of incorporating language priors in downstream dialog generation tasks. However, the performance of pre-trained models on the dialog task is not as optimal as expected. In this paper, we propose a Pre-trained Role Alternating Language model (PRAL), designed specifically for task-oriented conversational systems. We adopted (Wu et al., 2019) that models two speakers separately. We also design several techniques, such as start position randomization, knowledge distillation, and history discount to improve pre-training performance. We introduce a task-oriented dialog pretraining dataset by cleaning 13 existing data sets. We test PRAL on three different downstream tasks. The results show that PRAL performs better or on par with state-of-the-art methods.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2004.13835")
get_code_for_paper("2004.13835")
have("2004.13835")
The run record, dated, one paper per request, free:
curl https://syntology.ai/api/ran/2004.13835.json
A badge for a README (the split and the date, never a ratio):
[](https://syntology.ai/paper/2004.13835)
Connect an agent — have() is free.