We lifted 6 functions out of this paper's own repositories and ran 2 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| leixy20/scaffold | canonical | 2 of 6 |
| Function | Status | Where it lives |
|---|---|---|
| encode_image | Ran | leixy20/scaffold/call-api.py pointer only (licence: NONE) · get_code("181b183e1782d76e") |
| open_image | Ran | leixy20/scaffold/image_processor.py pointer only (licence: NONE) · get_code("89c0f11550ba932b") |
| dot_matrix_three_dimensional_single | Not yet run | leixy20/scaffold/image_processor.py pointer only (licence: NONE) · get_code("d03a25117d53b0d3") |
| dot_matrix_two_dimensional | Not yet run | leixy20/scaffold/image_processor.py pointer only (licence: NONE) · get_code("a7ec83d93ecf1cc6") |
| query_single_turn | Not yet run | leixy20/scaffold/call-api.py pointer only (licence: NONE) · get_code("a78eaa391081f627") |
| query_single_turn_and_save | Not yet run | leixy20/scaffold/call-api.py pointer only (licence: NONE) · get_code("5018d0f71b401df6") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
State-of-the-art Large Multi-Modal Models (LMMs) have demonstrated exceptional capabilities in vision-language tasks. Despite their advanced functionalities, the performances of LMMs are still limited in challenging scenarios that require complex reasoning with multiple levels of visual information. Existing prompting techniques for LMMs focus on either improving textual reasoning or leveraging tools for image preprocessing, lacking a simple and general visual prompting scheme to promote vision-language coordination in LMMs. In this work, we propose Scaffold prompting that scaffolds coordinates to promote vision-language coordination. Specifically, Scaffold overlays a dot matrix within the image as visual information anchors and leverages multi-dimensional coordinates as textual positional references. Extensive experiments on a wide range of challenging vision-language tasks demonstrate the superiority of Scaffold over GPT-4V with the textual CoT prompting. Our code is released in https://github.com/leixy20/Scaffold.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2402.12058")
get_code_for_paper("2402.12058")
have("2402.12058")
Connect an agent — have() is free.