Siqi Li, Jan Niehues, Danni Liu
We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Direct speech translation (ST) models often struggle with rare words. Incorrect translation of these words can have severe consequences, impacting translation quality and user trust. While rare word translation is inherently challenging for neural models due to sparse learning signals, real-world scenarios often allow access to translations of past recordings on similar topics. To leverage these valuable resources, we propose a retrieval-and-demonstration approach to enhance rare word translation accuracy in direct ST models. First, we adapt existing ST models to incorporate retrieved examples for rare word translation, which allows the model to benefit from prepended examples, similar to in-context learning. We then develop a cross-modal (speech-to-speech, speechto-text, text-to-text) retriever to locate suitable examples. We demonstrate that standard ST models can be effectively adapted to leverage examples for rare word translation, improving rare word translation accuracy over the baseline by 17.6% with gold examples and 8.5% with retrieved examples. Moreover, our speechto-speech retrieval approach outperforms other modalities and exhibits higher robustness to unseen speakers. Our code is publicly available 1 . Transformer Encoder (ST) 𝑢: 𝑢 ! : Transformer Decoder (ST) 𝑢: 𝑦 ! : …Aufhebens um Tipper.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2409.09009")
get_code_for_paper("2409.09009")
have("2409.09009")
The run record, dated, one paper per request, free:
curl https://syntology.ai/api/ran/2409.09009.json
A badge for a README (the split and the date, never a ratio):
[](https://syntology.ai/paper/2409.09009)
Connect an agent — have() is free.