Yang Shen, Errui Ding, Yazhou Yao, Yifan Sun, Xiu-Shen Wei, Yuxin Song, Jian Jin, Tao Yuan, Heyang Xu
We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Computer Vision (CV) has yet to fully achieve the zero-shot task generalization observed in Natural Language Processing (NLP), despite following many of the milestones established in NLP, such as large transformer models, extensive pre-training, and the auto-regression paradigm, among others. In this paper, we rethink the reality that CV adopts discrete and terminological task definitions (e.g., "image segmentation"), and conjecture it is a key barrier that hampers zeroshot task generalization. Our hypothesis is that without truly understanding previously-seen tasksdue to these terminological definitions-deep models struggle to generalize to novel tasks. To verify this, we introduce Explanatory Instructions, which provide an intuitive way to define CV task objectives through detailed linguistic transformations from input images to outputs. We create a large-scale dataset comprising 12 million "image input → explanatory instruction → output" triplets, and train an auto-regressive-based visionlanguage model (AR-based VLM) that takes both images and explanatory instructions as input.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2412.18525")
get_code_for_paper("2412.18525")
have("2412.18525")
The run record, dated, one paper per request, free:
curl https://syntology.ai/api/ran/2412.18525.json
A badge for a README (the split and the date, never a ratio):
[](https://syntology.ai/paper/2412.18525)
Connect an agent — have() is free.