Quoc Le, Minh-Thang Luong, Mohit Iyyer, Tu Vu, Grady Simon
We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Despite their recent successes in tackling many NLP tasks, large-scale pre-trained language models do not perform as well in few-shot settings where only a handful of training examples are available. To address this shortcoming, we propose STraTA, which stands for Self-Training with Task Augmentation, an approach that builds on two key ideas for effective leverage of unlabeled data. First, STraTA uses task augmentation, a novel technique that synthesizes a large amount of data for auxiliary-task fine-tuning from target-task unlabeled texts. Second, STraTA performs selftraining by further fine-tuning the strong base model created by task augmentation on a broad distribution of pseudo-labeled data. Our experiments demonstrate that STraTA can substantially improve sample efficiency across 12 fewshot benchmarks. Remarkably, on the SST-2 sentiment dataset, STraTA, with only 8 training examples per class, achieves comparable results to standard fine-tuning with 67K training examples. Our analyses reveal that task augmentation and self-training are both complementary and independently effective.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2109.06270")
get_code_for_paper("2109.06270")
have("2109.06270")
The run record, dated, one paper per request, free:
curl https://syntology.ai/api/ran/2109.06270.json
A badge for a README (the split and the date, never a ratio):
[](https://syntology.ai/paper/2109.06270)
Connect an agent — have() is free.