Yi Zhang, Dan Roth, Aaron Mueller, Saab Mansour, Salvatore Romeo, Elman Mansimov, Jason Krone
We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
In text classification tasks, useful information is encoded in the label names. Label semantic aware systems have leveraged this information for improved text classification performance during fine-tuning and prediction. However, use of label-semantics during pre-training has not been extensively explored. We therefore propose Label Semantic Aware Pre-training (LSAP) to improve the generalization and data efficiency of text classification systems. LSAP incorporates label semantics into pre-trained generative models (T5 in our case) by performing secondary pre-training on labeled sentences from a variety of domains. As domaingeneral pre-training requires large amounts of data, we develop a filtering and labeling pipeline to automatically create sentence-label pairs from unlabeled text. We perform experiments on intent (ATIS, Snips, TOPv2) and topic classification (AG News, Yahoo! Answers). LSAP obtains significant accuracy improvements over state-of-the-art models for few-shot text classification while maintaining performance comparable to state of the art in high-resource settings.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2204.07128")
get_code_for_paper("2204.07128")
have("2204.07128")
The run record, dated, one paper per request, free:
curl https://syntology.ai/api/ran/2204.07128.json
A badge for a README (the split and the date, never a ratio):
[](https://syntology.ai/paper/2204.07128)
Connect an agent — have() is free.