Karl Stratos, Wenzheng Zhang
We lifted 2 functions out of this paper's own repositories and ran 2 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| WenzhengZhang/hard-nce-el | canonical | 2 of 2 |
| Function | Status | Where it lives |
|---|---|---|
| evaluate | Ran | WenzhengZhang/hard-nce-el/main_retriever.py code served (permissive licence) · get_code("f9345bf90bc76e4e") |
| micro_eval | Ran | WenzhengZhang/hard-nce-el/main_reranker.py code served (permissive licence) · get_code("fd68bff6ce46ed21") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
The choice of negative examples is important in noise contrastive estimation. Recent works find that hard negatives-highest-scoring incorrect examples under the model-are effective in practice, but they are used without a formal justification. We develop analytical tools to understand the role of hard negatives. Specifically, we view the contrastive loss as a biased estimator of the gradient of the crossentropy loss, and show both theoretically and empirically that setting the negative distribution to be the model distribution results in bias reduction. We also derive a general form of the score function that unifies various architectures used in text retrieval. By combining hard negatives with appropriate score functions, we obtain strong results on the challenging task of zero-shot entity linking.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2104.06245")
get_code_for_paper("2104.06245")
have("2104.06245")
Connect an agent — have() is free.