Valeo Ai, Patrick Pérez, Mohammad Fahes, Andrei Bursuc, Kyutai, Raoul De Charette, Tuan-Hung Vu
We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. If it links a repository, it is listed below.
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
We introduce ProLIP, a simple and architecture-agnostic method for adapting contrastively pretrained visionlanguage models, such as CLIP [36], to few-shot classification. ProLIP fine-tunes the vision encoder's projection matrix with Frobenius norm regularization on its deviation from the pretrained weights. It achieves state-of-the-art performance on 11 few-shot classification benchmarks under both "few-shot validation" [23] and "validation-free" [42] settings. Moreover, by rethinking the non-linear CLIP-Adapter [13] through ProLIP's lens, we design a Regularized Linear Adapter (RLA) that performs better, requires no hyperparameter tuning, is less sensitive to learning rate values, and offers an alternative to ProLIP in black-box scenarios where model weights are inaccessible. Beyond fewshot classification, ProLIP excels in cross-dataset transfer, domain generalization, base-to-new class generalization, and test-time adaptation-where it outperforms prompt tuning while being an order of magnitude faster to train. Code is available at https://github.com/astravision/ProLIP.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2410.05270")
get_code_for_paper("2410.05270")
have("2410.05270")
Connect an agent — have() is free.