SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2304.04613 · EMNLP · 2023

On Evaluation of Bangla Word Analogies

Souvika Sarkar, Mousumi Akter, Shubhra Kanti, " Santu

arXiv · PDF · Open in the Atlas

Code that ran

We lifted 11 functions out of this paper's own repositories and ran 7 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.

RepositoryRoleRan
facebookresearch/fastText canonical 7 of 11
FunctionStatusWhere it lives
find_nearest_neighbor Ran facebookresearch/fastText/python/fasttext_module/fasttext/util/util.py
pointer only (licence: MIT) · get_code("5d48a5764718f390")
guess_target_name Ran facebookresearch/fastText/reduce_model.py
pointer only (licence: MIT) · get_code("c2a80677a6cb6044")
idx Ran facebookresearch/fastText/alignment/utils.py
pointer only (licence: MIT) · get_code("831f6e244ecea2bb")
procrustes Ran facebookresearch/fastText/alignment/utils.py
pointer only (licence: MIT) · get_code("6f28ff1eccefb093")
read_args Ran facebookresearch/fastText/python/fasttext_module/fasttext/FastText.py
pointer only (licence: MIT) · get_code("f697966cbe929024")
sqrt_eig Ran facebookresearch/fastText/alignment/unsup_align.py
pointer only (licence: MIT) · get_code("e4a10d52bc6d1601")
test Ran facebookresearch/fastText/python/fasttext_module/fasttext/util/util.py
pointer only (licence: MIT) · get_code("5c3bcf5ae2042c99")
getknn Not yet run facebookresearch/fastText/alignment/align.py
pointer only (licence: MIT) · get_code("f685d8c430dfac70")
load_vectors Not yet run facebookresearch/fastText/alignment/utils.py
pointer only (licence: MIT) · get_code("5b38e4065c54800a")
rcsls Not yet run facebookresearch/fastText/alignment/unsup_multialign.py
pointer only (licence: MIT) · get_code("d484b37352fc8959")
reduce_model Not yet run facebookresearch/fastText/python/fasttext_module/fasttext/util/util.py
pointer only (licence: MIT) · get_code("3ab9d330ea24c434")

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

This paper presents a benchmark dataset of Bangla word analogies for evaluating the quality of existing Bangla word embeddings. Despite being the 7 th largest spoken language in the world, Bangla is still a low-resource language and popular NLP models often struggle to perform well on Bangla data sets. Therefore, developing a robust evaluation set is crucial for benchmarking and guiding future research on improving Bangla word embeddings, which is currently missing. To address this issue, we introduce a new evaluation set of 16,678 unique word analogies in Bangla as well as a translated and curated version of the original Mikolov dataset (10,594 samples) in Bangla. Our experiments with different state-of-the-art embedding models reveal that current Bangla word embeddings struggle to achieve high accuracy on both data sets, demonstrating a significant gap in multilingual NLP research.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2304.04613")
get_code_for_paper("2304.04613")
have("2304.04613")

Connect an agent — have() is free.