Qian Li, Yulan He, Lin Gui, Hanqi Yan, Yanran Li, Hongjing Li
We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Recent years have witnessed increasing interests in prompt-based learning in which models can be trained on only a few annotated instances, making them suitable in low-resource settings. When using prompt-based learning for text classification, the goal is to use a pretrained language model (PLM) to predict a missing token in a pre-defined template given an input text, which can be mapped to a class label. However, PLMs built on the transformer architecture tend to generate similar output embeddings, making it difficult to discriminate between different class labels. The problem is further exacerbated when dealing with classification tasks involving many fine-grained class labels. In this work, we alleviate this information diffusion issue, i.e., different tokens share a large proportion of similar information after going through stacked multiple self-attention layers in a transformer, by proposing a calibration method built on feature transformations through rotation and scaling to map a PLMencoded embedding into a new metric space to guarantee the distinguishability of the resulting embeddings. Furthermore, we take the advantage of hyperbolic embeddings to capture the hierarchical relations among fine-grained classassociated token embedding by a coarse-to-fine metric learning strategy to enhance the distinguishability of the learned output embeddings. Extensive experiments on the three datasets under various settings demonstrate the effectiveness of our approach. 1 * Equal contribution.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2302.06198")
get_code_for_paper("2302.06198")
have("2302.06198")
The run record, dated, one paper per request, free:
curl https://syntology.ai/api/ran/2302.06198.json
A badge for a README (the split and the date, never a ratio):
[](https://syntology.ai/paper/2302.06198)
Connect an agent — have() is free.