Florian Metze, Siddharth Dalmia, Vikas Raunak, Vivek Gupta
We lifted 1 functions out of this paper's own repositories and ran 1 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| vyraun/long-tailed | canonical | 1 of 1 |
| Function | Status | Where it lives |
|---|---|---|
| get_idf_score | Ran | vyraun/long-tailed/analysis/normalize_last_layer.py code served (permissive licence) · get_code("170e2b9970e01fa2") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
State-of-the-art Neural Machine Translation (NMT) models struggle with generating lowfrequency tokens, tackling which remains a major challenge. The analysis of long-tailed phenomena in the context of structured prediction tasks is further hindered by the added complexities of search during inference. In this work, we quantitatively characterize such longtailed phenomena at two levels of abstraction, namely, token classification and sequence generation. We propose a new loss function, the Anti-Focal loss, to better adapt model training to the structural dependencies of conditional text generation by incorporating the inductive biases of beam search in the training process. We show the efficacy of the proposed technique on a number of Machine Translation (MT) datasets, demonstrating that it leads to significant gains over cross-entropy across different language pairs, especially on the generation of low-frequency words. We have released the code to reproduce our results. 1
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2010.04924")
get_code_for_paper("2010.04924")
have("2010.04924")
Connect an agent — have() is free.