SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2103.16590 · EMNLP · 2021

Evaluating the Morphosyntactic Well-formedness of Generated Texts

Graham Neubig, Shruti Rijhwani, Antonios Anastasopoulos, Yulia Tsvetkov, David Mortensen, Adithya Pratapa, Aditi Chaudhary

arXiv · PDF · Open in the Atlas

Code that ran

We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

Text generation systems are ubiquitous in natural language processing applications. However, evaluation of these systems remains a challenge, especially in multilingual settings. In this paper, we propose L'AMBRE -a metric to evaluate the morphosyntactic wellformedness of text using its dependency parse and morphosyntactic rules of the language. We present a way to automatically extract various rules governing morphosyntax directly from dependency treebanks. To tackle the noisy outputs from text generation systems, we propose a simple methodology to train robust parsers. We show the effectiveness of our metric on the task of machine translation through a diachronic study of systems translating into morphologically-rich languages. 1

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2103.16590")
get_code_for_paper("2103.16590")
have("2103.16590")

The run record, dated, one paper per request, free:

curl https://syntology.ai/api/ran/2103.16590.json

A badge for a README (the split and the date, never a ratio):

[![Syntology run record](https://syntology.ai/api/ran/2103.16590.svg)](https://syntology.ai/paper/2103.16590)

Connect an agent — have() is free.