Boxing Chen, Minpeng Liao, Kai Fan, Zhongqiang Huang, Yidong Chen, Xiaodong Shi, Biao Fu
We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
A popular approach to streaming speech translation is to employ a single offline model with a wait-k policy to support different latency requirements, which is simpler than training multiple online models with different latency constraints. However, there is a mismatch problem in using a model trained with complete utterances for streaming inference with partial input. We demonstrate that speech representations extracted at the end of a streaming input are significantly different from those extracted from a complete utterance. To address this issue, we propose a new approach called Future-Aware Streaming Translation (FAST) that adapts an offline ST model for streaming input. FAST includes a Future-Aware Inference (FAI) strategy that incorporates future context through a trainable masked embedding, and a Future-Aware Distillation (FAD) framework that transfers future context from an approximation of full speech to streaming input. Our experiments on the MuST-C EnDe, EnEs, and EnFr benchmarks show that FAST achieves better trade-offs between translation quality and latency than strong baselines. Extensive analyses suggest that our methods effectively alleviate the aforementioned mismatch problem between offline training and online inference. 1
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2303.07914")
get_code_for_paper("2303.07914")
have("2303.07914")
The run record, dated, one paper per request, free:
curl https://syntology.ai/api/ran/2303.07914.json
A badge for a README (the split and the date, never a ratio):
[](https://syntology.ai/paper/2303.07914)
Connect an agent — have() is free.