We lifted 12 functions out of this paper's own repositories and ran 11 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| lemmonation/fcl-nat | canonical | 11 of 12 |
| Function | Status | Where it lives |
|---|---|---|
| add_standard_attention_hparams | Ran | lemmonation/fcl-nat/tensor2tensor/layers/common_attention.py code served (permissive licence) · get_code("44e847faaba1328c") |
| attention_bias_without_self | Ran | lemmonation/fcl-nat/tensor2tensor/models/transformer_nat_cl_word.py code served (permissive licence) · get_code("1cca0cf169d3512e") |
| copy_batches | Ran | lemmonation/fcl-nat/tensor2tensor/models/transformer_nat_cl_word.py code served (permissive licence) · get_code("ff01c7e565eb434e") |
| hard_sigmoid | Ran | lemmonation/fcl-nat/tensor2tensor/layers/common_layers.py code served (permissive licence) · get_code("62078081af34b8ee") |
| hard_tanh | Ran | lemmonation/fcl-nat/tensor2tensor/layers/common_layers.py code served (permissive licence) · get_code("dfdf8307a155fa8b") |
| is_in_expr | Ran | lemmonation/fcl-nat/tensor2tensor/data_generators/algorithmic_math.py code served (permissive licence) · get_code("7e2bc85a4d4189a9") |
| lower_endian_to_number | Ran | lemmonation/fcl-nat/tensor2tensor/data_generators/algorithmic.py code served (permissive licence) · get_code("b1896a2536602d28") |
| random_expr | Ran | lemmonation/fcl-nat/tensor2tensor/data_generators/algorithmic_math.py code served (permissive licence) · get_code("916cc023d1d1ddbd") |
| random_expr_with_required_var | Ran | lemmonation/fcl-nat/tensor2tensor/data_generators/algorithmic_math.py code served (permissive licence) · get_code("0a76872f8ddc4b1e") |
| zipf_distribution | Ran | lemmonation/fcl-nat/tensor2tensor/data_generators/algorithmic.py code served (permissive licence) · get_code("68fe9a15fc61f6bd") |
| zipf_random_sample | Ran | lemmonation/fcl-nat/tensor2tensor/data_generators/algorithmic.py code served (permissive licence) · get_code("d0d6f2cc06f67920") |
| saturating_sigmoid | Not yet run | lemmonation/fcl-nat/tensor2tensor/layers/common_layers.py code served (permissive licence) · get_code("c53fad116e23fd5d") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Non-autoregressive translation (NAT) models remove the dependence on previous target tokens and generate all target tokens in parallel, resulting in significant inference speedup but at the cost of inferior translation accuracy compared to autoregressive translation (AT) models. Considering that AT models have higher accuracy and are easier to train than NAT models, and both of them share the same model configurations, a natural idea to improve the accuracy of NAT models is to transfer a well-trained AT model to an NAT model through fine-tuning. However, since AT and NAT models differ greatly in training strategy, straightforward fine-tuning does not work well. In this work, we introduce curriculum learning into fine-tuning for NAT. Specifically, we design a curriculum in the fine-tuning process to progressively switch the training from autoregressive generation to non-autoregressive generation. Experiments on four benchmark translation datasets show that the proposed method achieves good improvement (more than $1$ BLEU score) over previous NAT baselines in terms of translation accuracy, and greatly speed up (more than $10$ times) the inference process over AT baselines.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("1911.08717")
get_code_for_paper("1911.08717")
have("1911.08717")
Connect an agent — have() is free.