SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2108.00356 · 2021

Improving Social Meaning Detection with Pragmatic Masking and Surrogate Fine-Tuning

arXiv · PDF · Open in the Atlas

Code that ran

We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

Masked language models (MLMs) are pre-trained with a denoising objective that is in a mismatch with the objective of downstream fine-tuning. We propose pragmatic masking and surrogate fine-tuning as two complementing strategies that exploit social cues to drive pre-trained representations toward a broad set of concepts useful for a wide class of social meaning tasks. We test our models on $15$ different Twitter datasets for social meaning detection. Our methods achieve $2.34\%$ $F_1$ over a competitive baseline, while outperforming domain-specific language models pre-trained on large datasets. Our methods also excel in few-shot learning: with only $5\%$ of training data (severely few-shot), our methods enable an impressive $68.54\%$ average $F_1$. The methods are also language agnostic, as we show in a zero-shot setting involving six datasets from three different languages.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2108.00356")
get_code_for_paper("2108.00356")
have("2108.00356")

The run record, dated, one paper per request, free:

curl https://syntology.ai/api/ran/2108.00356.json

A badge for a README (the split and the date, never a ratio):

[![Syntology run record](https://syntology.ai/api/ran/2108.00356.svg)](https://syntology.ai/paper/2108.00356)

Connect an agent — have() is free.