We lifted 3 functions out of this paper's own repositories and ran 0 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| coetaur0/ESIM | pwc_unofficial | 0 of 3 |
| Function | Status | Where it lives |
|---|---|---|
| get_mask | Not yet run | coetaur0/ESIM/esim/utils.py code served (permissive licence) · get_code("4ea84e4fb7bd3780") |
| masked_softmax | Not yet run | coetaur0/ESIM/esim/utils.py code served (permissive licence) · get_code("9a7c5b85431b6c93") |
| sort_by_seq_lens | Not yet run | coetaur0/ESIM/esim/utils.py code served (permissive licence) · get_code("f020fb77f3123cb3") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
We create a new NLI test set that shows the deficiency of state-of-the-art models in inferences that require lexical and world knowledge. The new examples are simpler than the SNLI test set, containing sentences that differ by at most one word from sentences in the training set. Yet, the performance on the new test set is substantially worse across systems trained on SNLI, demonstrating that these systems are limited in their generalization ability, failing to capture many simple inferences.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("1805.02266")
get_code_for_paper("1805.02266")
have("1805.02266")
Connect an agent — have() is free.