SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2407.03056 · 2024

Improving Zero-shot Generalization of Learned Prompts via Unsupervised Knowledge Distillation

arXiv · PDF · Open in the Atlas

Code that ran

We lifted 8 functions out of this paper's own repositories and ran 6 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.

RepositoryRoleRan
miccunifi/kdpl canonical 6 of 8
FunctionStatusWhere it lives
basic_clean Ran miccunifi/kdpl/src/clip/simple_tokenizer.py
pointer only (licence: NONE) · get_code("98f385d847636a3e")
compute_ci95 Ran miccunifi/kdpl/src/parse_test_res.py
pointer only (licence: NONE) · get_code("ba26afd892405335")
get_K_max Ran miccunifi/kdpl/src/trainers/kdpl_utils.py
pointer only (licence: NONE) · get_code("ff34933f02b58a29")
get_pairs Ran miccunifi/kdpl/src/clip/simple_tokenizer.py
pointer only (licence: NONE) · get_code("d919ae32e5e4e616")
sampling Ran miccunifi/kdpl/src/trainers/kdpl_utils.py
pointer only (licence: NONE) · get_code("e49b1a864e601506")
whitespace_clean Ran miccunifi/kdpl/src/clip/simple_tokenizer.py
pointer only (licence: NONE) · get_code("9542161e9640b858")
build_model Not yet run miccunifi/kdpl/src/clip/model.py
pointer only (licence: NONE) · get_code("62be2590c6547063")
load Not yet run miccunifi/kdpl/src/clip/clip.py
pointer only (licence: NONE) · get_code("0695164c9caf0b87")

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

Vision-Language Models (VLMs) demonstrate remarkable zero-shot generalization to unseen tasks, but fall short of the performance of supervised methods in generalizing to downstream tasks with limited data. Prompt learning is emerging as a parameter-efficient method for adapting VLMs, but state-of-the-art approaches require annotated samples. In this paper we propose a novel approach to prompt learning based on unsupervised knowledge distillation from more powerful models. Our approach, which we call Knowledge Distillation Prompt Learning (KDPL), can be integrated into existing prompt learning techniques and eliminates the need for labeled examples during adaptation. Our experiments on more than ten standard benchmark datasets demonstrate that KDPL is very effective at improving generalization of learned prompts for zero-shot domain generalization, zero-shot cross-dataset generalization, and zero-shot base-to-novel class generalization problems. KDPL requires no ground-truth labels for adaptation, and moreover we show that even in the absence of any knowledge of training class names it can be used to effectively transfer knowledge. The code is publicly available at https://github.com/miccunifi/KDPL.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2407.03056")
get_code_for_paper("2407.03056")
have("2407.03056")

Connect an agent — have() is free.