Aaron Courville, Devon Hjelm, Siva Reddy, Alessandro Sordoni, Dzmitry Bahdanau, Arian Hosseini
We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Negation is a core construction in natural language. Despite being very successful on many tasks, state-of-the-art pre-trained language models often handle negation incorrectly. To improve language models in this regard, we propose to augment the language modeling objective with an unlikelihood objective that is based on negated generic sentences from a raw text corpus. By training BERT with the resulting combined objective we reduce the mean top 1 error rate to 4% on the negated LAMA dataset. We also see some improvements on the negated NLI benchmarks. Top 3 words with log probs from BERT Top 3 words with log probs from BERTNOT iOS is developed by [MASK]. Apple (-1.8), Google (-2.6), Microsoft (-2.8) Apple (-1.8), Google (-2.5), Microsoft (-2.7) iOS is not developed by [MASK]. Apple (-1.8), Google (-2.6), Microsoft (-2.8) Microsoft (-1.8), Google (-2.4), Apple (-3.1) The majority of the amazon forest is in [MASK]. Brazil (-2.6), Bolivia (-2.7), Madagascar (-3.1) Brazil (-2.9), Bolivia (-3.1), Mexico (-3.2) The majority of the amazon forest is not in [MASK]. cultivation (-1.0), Brazil (-3.5), Mexico (-3.5) cultivation (-2.0), Mexico (-4.1), France (-4.3) Charles Nodier died in [MASK]. Paris (-1.35), Rome (-3.2), office (-3.4) Paris (-1.5), Rome (-3.3), France (-3.6) Charles Nodier did not die in [MASK]. Paris (-2.4), office (-2.7), France (-2.8) vain (-3.5), error (-4.0), doubt (-4.5) Mac OS is developed by [MASK]. Apple (-1.9), Microsoft (-2.0), Intel (-2.0) Apple (-2.0), Microsoft (-2.0), Intel (-2.1) Mac OS is not developed by [MASK]. Apple (-1.3), Microsoft (-1.5), IBM (-2.3) Microsoft (-2.1), IBM (-2.7), itself (-3.4)
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2105.03519")
get_code_for_paper("2105.03519")
have("2105.03519")
The run record, dated, one paper per request, free:
curl https://syntology.ai/api/ran/2105.03519.json
A badge for a README (the split and the date, never a ratio):
[](https://syntology.ai/paper/2105.03519)
Connect an agent — have() is free.