Chengyu Wang, Jun Huang, Xiaofeng He, Minghui Qiu
We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Pre-trained neural language models bring significant improvement for various NLP tasks, by fine-tuning the models on task-specific training sets. During fine-tuning, the parameters are initialized from pre-trained models directly, which ignores how the learning process of similar NLP tasks in different domains is correlated and mutually reinforced. In this paper, we propose an effective learning procedure named Meta Fine-Tuning (MFT), serving as a meta-learner to solve a group of similar NLP tasks for neural language models. Instead of simply multi-task training over all the datasets, MFT only learns from typical instances of various domains to acquire highly transferable knowledge. It further encourages the language model to encode domaininvariant representations by optimizing a series of novel domain corruption loss functions. After MFT, the model can be fine-tuned for each domain with better parameter initialization and higher generalization ability. We implement MFT upon BERT to solve several multi-domain text mining tasks. Experimental results confirm the effectiveness of MFT and its usefulness for few-shot learning. 1
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2003.13003")
get_code_for_paper("2003.13003")
have("2003.13003")
The run record, dated, one paper per request, free:
curl https://syntology.ai/api/ran/2003.13003.json
A badge for a README (the split and the date, never a ratio):
[](https://syntology.ai/paper/2003.13003)
Connect an agent — have() is free.