Omri Abend, Nathan Schneider, Vivek Srikumar, Taelin Karidi, Yichu Zhou
We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
We present a method for exploring regions around individual points in a contextualized vector space (particularly, BERT space), as a way to investigate how these regions correspond to word senses. By inducing a contextualized "pseudoword" as a stand-in for a static embedding in the input layer, and then performing masked prediction of a word in the sentence, we are able to investigate the geometry of the BERT-space in a controlled manner around individual instances. Using our method on a set of carefully constructed sentences targeting ambiguous English words, we find substantial regularity in the contextualized space, with regions that correspond to distinct word senses; but between these regions there are occasionally "sense voids"-regions that do not correspond to any intelligible sense. 1
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2109.11491")
get_code_for_paper("2109.11491")
have("2109.11491")
The run record, dated, one paper per request, free:
curl https://syntology.ai/api/ran/2109.11491.json
A badge for a README (the split and the date, never a ratio):
[](https://syntology.ai/paper/2109.11491)
Connect an agent — have() is free.