Barry Haddow, Christos Baziotis, Alexandra Birch
We lifted 6 functions out of this paper's own repositories and ran 6 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| cbaziotis/lm-prior-for-nmt | canonical | 6 of 6 |
| Function | Status | Where it lives |
|---|---|---|
| baseline | Ran | cbaziotis/lm-prior-for-nmt/configs/rnn/generate_nmt_experiments.py code served (permissive licence) · get_code("70ea29561d623be2") |
| baseline | Ran | cbaziotis/lm-prior-for-nmt/configs/transformer/generate_nmt_experiments.py code served (permissive licence) · get_code("de1492b5da3b7bb5") |
| fusion | Ran | cbaziotis/lm-prior-for-nmt/configs/rnn/generate_nmt_experiments.py code served (permissive licence) · get_code("baf87cd1b3bd046c") |
| fusion | Ran | cbaziotis/lm-prior-for-nmt/configs/transformer/generate_nmt_experiments.py code served (permissive licence) · get_code("0471dc0474e44f8d") |
| get_name | Ran | cbaziotis/lm-prior-for-nmt/configs/rnn/generate_nmt_experiments.py code served (permissive licence) · get_code("cb3d7ac764f53331") |
| get_name | Ran | cbaziotis/lm-prior-for-nmt/configs/transformer/generate_nmt_experiments.py code served (permissive licence) · get_code("0f1a2cddb04257e0") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
The scarcity of large parallel corpora is an important obstacle for neural machine translation. A common solution is to exploit the knowledge of language models (LM) trained on abundant monolingual data. In this work, we propose a novel approach to incorporate a LM as prior in a neural translation model (TM). Specifically, we add a regularization term, which pushes the output distributions of the TM to be probable under the LM prior, while avoiding wrong predictions when the TM "disagrees" with the LM. This objective relates to knowledge distillation, where the LM can be viewed as teaching the TM about the target language. The proposed approach does not compromise decoding speed, because the LM is used only at training time, unlike previous work that requires it during inference. We present an analysis on the effects that different methods have on the distributions of the TM. Results on two low-resource machine translation datasets show clear improvements even with limited monolingual data.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2004.14928")
get_code_for_paper("2004.14928")
have("2004.14928")
Connect an agent — have() is free.