Christopher Manning, Shikhar Murty, Xinran Zhao
We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
While advances in pre-training have led to dramatic improvements in few-shot learning of NLP tasks, there is limited understanding of what drives successful few-shot adaptation in datasets. In particular, given a new dataset and a pre-trained model, what properties of the dataset make it few-shot learnable and are these properties independent of the specific adaptation techniques used? We consider an extensive set of recent few-shot learning methods, and show that their performance across a large number of datasets is highly correlated, showing that few-shot hardness may be intrinsic to datasets, for a given pre-trained model. To estimate intrinsic few-shot hardness, we then propose a simple and lightweight metric called Spread that captures the intuition that fewshot learning is made possible by exploiting feature-space invariances between training and test samples. Our metric better accounts for few-shot hardness compared to existing notions of hardness, and is ~8-100x faster to compute.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2211.09113")
get_code_for_paper("2211.09113")
have("2211.09113")
The run record, dated, one paper per request, free:
curl https://syntology.ai/api/ran/2211.09113.json
A badge for a README (the split and the date, never a ratio):
[](https://syntology.ai/paper/2211.09113)
Connect an agent — have() is free.