Xiaodan Liang, Hang Xu, Xiwen Liang, Fengda Zhu, Lingling Li
We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Vision-language navigation (VLN) is a challenging task due to its large searching space in the environment. To address this problem, previous works have proposed some methods of fine-tuning a large model that pretrained on large-scale datasets. However, the conventional fine-tuning methods require extra human-labeled navigation data and lack self-exploration capabilities in environments, which hinders their generalization of unseen scenes. To improve the ability of fast cross-domain adaptation, we propose Prompt-based Environmental Self-exploration (ProbES), which can self-explore the environments by sampling trajectories and automatically generates structured instructions via a large-scale cross-modal pretrained model (CLIP). Our method fully utilizes the knowledge learned from CLIP to build an in-domain dataset by self-exploration without human labeling. Unlike the conventional approach of fine-tuning, we introduce prompt-based learning to achieve fast adaptation for language embeddings, which substantially improves the learning efficiency by leveraging prior knowledge. By automatically synthesizing trajectoryinstruction pairs in any environment without human supervision and efficient prompt-based learning, our model can adapt to diverse visionlanguage navigation tasks, including VLN and REVERIE. Both qualitative and quantitative results show that our ProbES significantly improves the generalization ability of the navigation model * .
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2203.04006")
get_code_for_paper("2203.04006")
have("2203.04006")
The run record, dated, one paper per request, free:
curl https://syntology.ai/api/ran/2203.04006.json
A badge for a README (the split and the date, never a ratio):
[](https://syntology.ai/paper/2203.04006)
Connect an agent — have() is free.