SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2407.01003 · 2024

EMBEDDED VISUAL PROMPT TUNING

Lei Ma, Guoqi Li, Qing Zhao, Lei, † Lei, Shenghao Xie, Wenqiang Zu

arXiv · PDF · Open in the Atlas

Code that ran

We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

Parameter-efficient fine-tuning (PEFT) methods aim to adapt large scale pre-trained models to new domains by updating a small portion of parameters to reduce computational overhead. However, the effectiveness of these PEFT methods, especially in cross-domain few-shot scenarios, e.g., medical images, has not been fully explored. This paper facilitate the study of the performance of PEFT when fine-tuning pre-trained models on medical images. Furthermore, to alleviate the limitations of prompt introducing ways and approximation capabilities on Transformer architectures of mainstream prompt tuning methods, we propose the Embedded Prompt Tuning (EPT) method by embedding prompt tokens into the expanded channels. We also find that there are anomalies in the feature space distribution of pre-trained models during pre-training process, asnd prompt tuning can help mitigate this negative impact. To explain this phenomenon, we also introduce a novel perspective to understand prompt tuning: Prompt tuning is a distribution calibrator. And we support it by analyzing patchwise scaling and feature separation operations contained in EPT. Our experiments show that EPT outperforms several state-of-the-art fine-tuning methods by a significant margin on few-shot medical image classification, and completes the fine-tuning process within highly competitive time, indicating EPT is an effective PEFT method. The source code is available at github.com/zuwenqiang/EPT.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2407.01003")
get_code_for_paper("2407.01003")
have("2407.01003")

The run record, dated, one paper per request, free:

curl https://syntology.ai/api/ran/2407.01003.json

A badge for a README (the split and the date, never a ratio):

[![Syntology run record](https://syntology.ai/api/ran/2407.01003.svg)](https://syntology.ai/paper/2407.01003)

Connect an agent — have() is free.