SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2006.02014 · ACL · 2020

Norm-Based Curriculum Learning for Neural Machine Translation

Xuebo Liu, Derek Wong, Lidia Chao, Nlp, Houtim Lai

arXiv · PDF · Open in the Atlas

Code that ran

We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

A neural machine translation (NMT) system is expensive to train, especially with highresource settings. As the NMT architectures become deeper and wider, this issue gets worse and worse. In this paper, we aim to improve the efficiency of training an NMT by introducing a novel norm-based curriculum learning method. We use the norm (aka length or module) of a word embedding as a measure of 1) the difficulty of the sentence, 2) the competence of the model, and 3) the weight of the sentence. The normbased sentence difficulty takes the advantages of both linguistically motivated and modelbased sentence difficulties. It is easy to determine and contains learning-dependent features. The norm-based model competence makes NMT learn the curriculum in a fully automated way, while the norm-based sentence weight further enhances the learning of the vector representation of the NMT. Experimental results for the WMT'14 English-German and WMT'17 Chinese-English translation tasks demonstrate that the proposed method outperforms strong baselines in terms of BLEU score (+1.17/+1.56) and training speedup (2.22x/3.33x).

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2006.02014")
get_code_for_paper("2006.02014")
have("2006.02014")

The run record, dated, one paper per request, free:

curl https://syntology.ai/api/ran/2006.02014.json

A badge for a README (the split and the date, never a ratio):

[![Syntology run record](https://syntology.ai/api/ran/2006.02014.svg)](https://syntology.ai/paper/2006.02014)

Connect an agent — have() is free.