Percy Liang, Robin Jia, Chris Donahue, Mina Lee, Alexander Iyabor
We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
We release a new benchmark for lexical substitution, the task of finding appropriate substitutes for a target word in a context. For writing, lexical substitution systems can assist humans by suggesting words that humans cannot easily think of. However, existing benchmarks depend on human recall as the only source of data, and therefore lack coverage of the substitutes that would be most helpful to humans. Furthermore, annotators often provide substitutes of low quality, which are not actually appropriate in the given context. We collect higher-coverage and higher-quality data by framing lexical substitution as a classification problem, guided by the intuition that it is easier for humans to judge the appropriateness of candidate substitutes than conjure them from memory. To this end, we use a contextfree thesaurus to produce candidates and rely on human judgement to determine contextual appropriateness. Compared to the previous largest benchmark, our SWORDS benchmark has 3x as many substitutes per target word for the same level of quality, and its substitutes are 1.4x more appropriate (based on human judgement) for the same number of substitutes. * Equal contribution.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2106.04102")
get_code_for_paper("2106.04102")
have("2106.04102")
The run record, dated, one paper per request, free:
curl https://syntology.ai/api/ran/2106.04102.json
A badge for a README (the split and the date, never a ratio):
[](https://syntology.ai/paper/2106.04102)
Connect an agent — have() is free.