Hong Qian, Xuanjing Huang, Yunhua Zhou, Zhengfu He, Tianxiang ♢♡, ♢♡ Xipeng, Qiu ♢♡
We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Most downstream adaptation methods tune all or part of the parameters of pre-trained models (PTMs) through gradient descent, where the tuning cost increases linearly with the growth of the model size. By contrast, gradient-free methods only require the forward computation of the PTM to tune the prompt, retaining the benefits of efficient tuning and deployment. Though, past work on gradient-free tuning often introduces gradient descent to seek a good initialization of prompt and lacks versatility across tasks and PTMs. In this paper, we present BBTv2, an improved version of Black-Box Tuning (Sun et al., 2022b), to drive PTMs for few-shot learning. We prepend continuous prompts to every layer of the PTM and propose a divide-and-conquer gradient-free algorithm to optimize the prompts at different layers alternately. Extensive experiments across various tasks and PTMs show that BBTv2 can achieve comparable performance to full model tuning and state-of-the-art parameter-efficient methods (e.g., Adapter, LoRA, BitFit, etc.) under few-shot settings while maintaining much fewer tunable parameters.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2205.11200")
get_code_for_paper("2205.11200")
have("2205.11200")
The run record, dated, one paper per request, free:
curl https://syntology.ai/api/ran/2205.11200.json
A badge for a README (the split and the date, never a ratio):
[](https://syntology.ai/paper/2205.11200)
Connect an agent — have() is free.