Graham Neubig, Junjie Hu, Mandela Patrick, Po-Yao Huang, Florian Metze, Alexander Hauptmann
We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
This paper studies zero-shot cross-lingual transfer of vision-language models. Specifically, we focus on multilingual text-tovideo search and propose a Transformer-based model that learns contextual multilingual multimodal embeddings. Under a zero-shot setting, we empirically demonstrate that performance degrades significantly when we query the multilingual text-video model with non-English sentences. To address this problem, we introduce a multilingual multimodal pre-training strategy, and collect a new multilingual instructional video dataset (Multi-HowTo100M) for pre-training. Experiments on VTT show that our method significantly improves video search in non-English languages without additional annotations. Furthermore, when multilingual annotations are available, our method outperforms recent baselines by a large margin in multilingual text-to-video search on VTT and VATEX; as well as in multilingual text-to-image search on Multi30K.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2103.08849")
get_code_for_paper("2103.08849")
have("2103.08849")
The run record, dated, one paper per request, free:
curl https://syntology.ai/api/ran/2103.08849.json
A badge for a README (the split and the date, never a ratio):
[](https://syntology.ai/paper/2103.08849)
Connect an agent — have() is free.