Mayank Singh, Shruti Singh
We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Language models are increasingly becoming popular in AI-powered scientific IR systems. This paper evaluates popular scientific language models in handling (i) short-query texts and (ii) textual neighbors. Our experiments showcase the inability to retrieve relevant documents for a short-query text even under the most relaxed conditions. Additionally, we leverage textual neighbors, generated by small perturbations to the original text, to demonstrate that not all perturbations lead to close neighbors in the embedding space. Further, an exhaustive categorization yields several classes of orthographically and semantically related, partially related and completely unrelated neighbors. Retrieval performance turns out to be more influenced by the surface form rather than the semantics of the text.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2203.15364")
get_code_for_paper("2203.15364")
have("2203.15364")
The run record, dated, one paper per request, free:
curl https://syntology.ai/api/ran/2203.15364.json
A badge for a README (the split and the date, never a ratio):
[](https://syntology.ai/paper/2203.15364)
Connect an agent — have() is free.