Nicu Sebe, Zhengyu Zhao, Songlong Xing
We lifted 4 functions out of this paper's own repositories and ran 2 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| Sxing2/CLIP-Test-time-Counterattacks | — | 2 of 3 |
| copy not recorded | — | 0 of 1 |
| Function | Status | Where it lives |
|---|---|---|
| clip_img_preprocessing | Ran | Sxing2/CLIP-Test-time-Counterattacks/code/attacks.py pointer only (licence: NONE) · get_code("29a9e9c6629ed618") |
| normalize | Ran | Sxing2/CLIP-Test-time-Counterattacks/code/attacks.py pointer only (licence: NONE) · get_code("dd42cbeb18af19eb") |
| attack_unlabelled_opp | Not yet run | Sxing2/CLIP-Test-time-Counterattacks/code/attacks.py pointer only (licence: NONE) · get_code("f5dcab1b5724f573") |
| clamp | Not yet run | this paper's copy was not recorded; identical code first harvested from csdongxian/AWP pointer only · get_code("8a93e041134b597a") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Despite its prevalent use in image-text matching tasks in a zero-shot manner, CLIP has been shown to be highly vulnerable to adversarial perturbations added onto images. Recent studies propose to finetune the vision encoder of CLIP with adversarial samples generated on the fly, and show improved robustness against adversarial attacks on a spectrum of downstream datasets, a property termed as zero-shot robustness. In this paper, we show that malicious perturbations that seek to maximise the classification loss lead to 'falsely stable' images, and propose to leverage the pre-trained vision encoder of CLIP to counterattack such adversarial images during inference to achieve robustness. Our paradigm is simple and training-free, providing the first method to defend CLIP from adversarial attacks at test time, which is orthogonal to existing methods aiming to boost zero-shot adversarial robustness of CLIP. We conduct experiments across 16 classification datasets, and demonstrate stable and consistent gains compared to test-time defence methods adapted from existing adversarial robustness studies that do not rely on external networks, without noticeably impairing performance on clean images. We also show that our paradigm can be employed on CLIP models that have been adversarially finetuned to further enhance their robustness at test time. Our code is available here.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2503.03613")
get_code_for_paper("2503.03613")
have("2503.03613")
Connect an agent — have() is free.