SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2406.20054 · EMNLP · 2024

To Word Senses and Beyond: Inducing Concepts with Contextualized Language Models

Pascal Denis, Mikaela Keller, Bastien Liétard

arXiv · PDF · Open in the Atlas

Code that ran

We lifted 4 functions out of this paper's own repositories and ran 2 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.

RepositoryRoleRan
blietard/concept-induction canonical 2 of 4
FunctionStatusWhere it lives
get_algo_class Ran blietard/concept-induction/utilities/clustering.py
code served (permissive licence) · get_code("452f936a6bdf7b80")
verif_layers_arg Ran blietard/concept-induction/utilities/languagemodel.py
code served (permissive licence) · get_code("3e405281884794dc")
get_tokenizer Not yet run blietard/concept-induction/utilities/languagemodel.py
code served (permissive licence) · get_code("81b99eca69445465")
initialize_models Not yet run blietard/concept-induction/utilities/languagemodel.py
code served (permissive licence) · get_code("e29e021d7ad378d7")

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

Polysemy and synonymy are two crucial interrelated facets of lexical ambiguity. While both phenomena are widely documented in lexical resources and have been studied extensively in NLP, leading to dedicated systems, they are often being considered independently in practictal problems. While many tasks dealing with polysemy (e.g. Word Sense Disambiguiation or Induction) highlight the role of word's senses, the study of synonymy is rooted in the study of concepts, i.e. meanings shared across the lexicon. In this paper, we introduce Concept Induction, the unsupervised task of learning a soft clustering among words that defines a set of concepts directly from data. This task generalizes Word Sense Induction. We propose a bi-level approach to Concept Induction that leverages both a local lemma-centric view and a global cross-lexicon view to induce concepts. We evaluate the obtained clustering on SemCor's annotated data and obtain good performance (BCubed F 1 above 0.60). We find that the local and the global levels are mutually beneficial to induce concepts and also senses in our setting. Finally, we create static embeddings representing our induced concepts and use them on the Word-in-Context task, obtaining competitive performance with the State-ofthe-Art.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2406.20054")
get_code_for_paper("2406.20054")
have("2406.20054")

Connect an agent — have() is free.