SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2303.14773 · CVPR · 2023

BlackVIP: Black-Box Visual Prompting for Robust Transfer Learning

Changdae Oh, Yongtaek Lim, Kyungwoo Song, Hyeji Hwang, Hee-Young Lee, Geunyoung Jung, Jiyoung Jung, Hosik Choi

arXiv · PDF · Open in the Atlas

Code that ran

We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

With the surge of large-scale pre-trained models (PTMs), fine-tuning these models to numerous downstream tasks becomes a crucial problem. Consequently, parameter efficient transfer learning (PETL) of large models has grasped huge attention. While recent PETL methods showcase impressive performance, they rely on optimistic assumptions: 1) the entire parameter set of a PTM is available, and 2) a sufficiently large memory capacity for the fine-tuning is equipped. However, in most real-world applications, PTMs are served as a black-box API or proprietary software without explicit parameter accessibility. Besides, it is hard to meet a large memory requirement for modern PTMs. In this work, we propose black-box visual prompting (Black-VIP), which efficiently adapts the PTMs without knowledge about model architectures and parameters. Black-VIP has two components; 1) Coordinator and 2) simultaneous perturbation stochastic approximation with gradient correction (SPSA-GC). The Coordinator designs inputdependent image-shaped visual prompts, which improves few-shot adaptation and robustness on distribution/location shift. SPSA-GC efficiently estimates the gradient of a target model to update Coordinator. Extensive experiments on 16 datasets demonstrate that BlackVIP enables robust adaptation to diverse domains without accessing PTMs' parameters, with minimal memory requirements.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2303.14773")
get_code_for_paper("2303.14773")
have("2303.14773")

The run record, dated, one paper per request, free:

curl https://syntology.ai/api/ran/2303.14773.json

A badge for a README (the split and the date, never a ratio):

[![Syntology run record](https://syntology.ai/api/ran/2303.14773.svg)](https://syntology.ai/paper/2303.14773)

Connect an agent — have() is free.