SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 1808.07561 · 2018

Training Deeper Neural Machine Translation Models with Transparent Attention

arXiv · PDF · Open in the Atlas

Code that ran

We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

While current state-of-the-art NMT models, such as RNN seq2seq and Transformers, possess a large number of parameters, they are still shallow in comparison to convolutional models used for both text and vision applications. In this work we attempt to train significantly (2-3x) deeper Transformer and Bi-RNN encoders for machine translation. We propose a simple modification to the attention mechanism that eases the optimization of deeper models, and results in consistent gains of 0.7-1.1 BLEU on the benchmark WMT'14 English-German and WMT'15 Czech-English tasks for both architectures.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("1808.07561")
get_code_for_paper("1808.07561")
have("1808.07561")

The run record, dated, one paper per request, free:

curl https://syntology.ai/api/ran/1808.07561.json

A badge for a README (the split and the date, never a ratio):

[![Syntology run record](https://syntology.ai/api/ran/1808.07561.svg)](https://syntology.ai/paper/1808.07561)

Connect an agent — have() is free.