SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2006.06202 · ACL · 2020

A Monolingual Approach to Contextualized Word Embeddings for Mid-Resource Languages

Benoît Sagot, Pedro Ortiz Suárez, Laurent Romary

arXiv · PDF · Open in the Atlas

Code that ran

We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. We have no repository linked to it either. That is a statement about our graph, not about the paper.

Abstract

We use the multilingual OSCAR corpus, extracted from Common Crawl via language classification, filtering and cleaning, to train monolingual contextualized word embeddings (ELMo) for five mid-resource languages. We then compare the performance of OSCARbased and Wikipedia-based ELMo embeddings for these languages on the part-ofspeech tagging and parsing tasks. We show that, despite the noise in the Common-Crawlbased OSCAR data, embeddings trained on OSCAR perform much better than monolingual embeddings trained on Wikipedia. They actually equal or improve the current state of the art in tagging and parsing for all five languages. In particular, they also improve over multilingual Wikipedia-based contextual embeddings (multilingual BERT), which almost always constitutes the previous state of the art, thereby showing that the benefit of a larger, more diverse corpus surpasses the crosslingual benefit of multilingual embedding architectures.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2006.06202")
get_code_for_paper("2006.06202")
have("2006.06202")

The run record, dated, one paper per request, free:

curl https://syntology.ai/api/ran/2006.06202.json

A badge for a README (the split and the date, never a ratio):

[![Syntology run record](https://syntology.ai/api/ran/2006.06202.svg)](https://syntology.ai/paper/2006.06202)

Connect an agent — have() is free.