SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2402.06659 · NeurIPS · 2024

Shadowcast: Stealthy Data Poisoning Attacks against Vision-Language Models

Tom Goldstein, Furong Huang, Manli Shu, Ning Yu, Yanchao Sun, Yuancheng Xu, Jiarui Yao, Zichu Wu

arXiv · PDF · Open in the Atlas

Code that ran

12 of 15 samples (functions and classes) lifted from this paper's linked repositories executed on a synthesized input in Syntology's jail (0 of them constructed an object rather than computing a result; 5 honoured the contract we drafted, 0 violated it and 0 have no contract verdict; 7 failures of our instrument; 3 no recorded run), measured 2026-11-22; repository links from the Papers with Code archive attributed under CC BY-SA, no affiliation; bodies inline only under a permissive licence as GitHub detects it. Not a reproduction of the paper's results.

"Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results. This record, dated, as JSON: /api/ran/2402.06659.json · badge: /api/ran/2402.06659.svg

Each repository's role is a label: a classifier's output recorded on the link, or, where no role is recorded, what the Papers with Code archive marked. The line under it names its basis. Neither says who wrote the code.

RepositoryRoleRan
umd-huang-lab/VLM-Poisoning canonical (classifier label)
classifier basis: census
11 ran · 3 no recorded run
umd-huang-lab/vlm-poisoning canonical (classifier label)
classifier basis: classifier_2025
1 ran · 0 no recorded run
FunctionStatusWhere it lives
chroma_subsampling Ran
ran; our contract draft was wrong, not the code
umd-huang-lab/VLM-Poisoning/DiffJPEG/compression.py
code served (permissive licence) · get_code("d0a996dc0d594c8a")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned tuple. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn

torch.manual_seed(0)

# Create a small batch of images with shape (batch, height, width, 3)
# Using distinct sizes: batch=2, height=4, width=6, channels=3
image = torch.randn(2, 4, 6, 3)

ARGS = {
    "image": image
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type tuple — a tuple output has no array shape of its own; none is stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
computeIoU Ran
ran; the contract check could not run (RAISES, DEP_MISSING or TIMEOUT)
umd-huang-lab/VLM-Poisoning/MiniGPT-4/demo_v2.py
code served (permissive licence) · get_code("f8e4c1c5aac977d6")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1) and returned float, ndim 0, dtype family float, shape () as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch

torch.manual_seed(0)

# The function computeIoU takes two bounding boxes
# Each bbox is a tuple/list of 4 elements: (x1, y1, x2, y2)
# representing coordinates of opposite corners

bbox1 = (10, 20, 50, 80)
bbox2 = (30, 40, 70, 100)

ARGS = {
    "bbox1": bbox1,
    "bbox2": bbox2,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type float, ndim 0, dtype family float, shape () — shape () as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
diff_round Ran
ran; our contract draft was wrong, not the code
umd-huang-lab/VLM-Poisoning/DiffJPEG/utils.py
code served (permissive licence) · get_code("37251df462837488")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned Tensor, ndim 2, dtype family float, shape (2, 4) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch

torch.manual_seed(0)

# The function diff_round takes a single tensor parameter x
# and performs differentiable rounding using torch operations.
# We need to provide a small tensor for testing.

x = torch.randn(2, 4, requires_grad=True)

ARGS = {
    "x": x
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Tensor, ndim 2, dtype family float, shape (2, 4) — shape (2, 4) as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
embedding_attack_Linf Ran
ran; the contract check could not run (RAISES, DEP_MISSING or TIMEOUT)
umd-huang-lab/vlm-poisoning/poison_llava.py
code served (permissive licence) · get_code("e57b7b37b157bb77")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1) and returned tuple. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn
from torchvision import transforms

torch.manual_seed(0)

# Define a minimal image encoder that takes normalized images and returns embeddings
class MinimalImageEncoder(nn.Module):
    def __init__(self, embedding_dim=8):
        super().__init__()
        self.embedding_dim = embedding_dim
        self.conv = nn.Conv2d(3, embedding_dim, kernel_size=1)
    
    def forward(self, x):
        # x shape: [bs, 3, H, W]
        # Output shape: [bs, embedding_dim]
        out = self.conv(x)  # [bs, embedding_dim, H, W]
        out = out.mean(dim=[2, 3])  # Global average pooling
        return out

# Define a minimal embedding distance function
def minimal_emb_dist(emb1, emb2):
    # emb1, emb2: [bs, embedding_dim]
    # Return: [bs] - pairwise L2 distances
    return torch.norm(emb1 - emb2, dim=1)

# Define a minimal differentiable augmentation
class MinimalDiffAug(nn.Module):
    def forward(self, x):
        # Simple identity augmentation for testing
        return x

# Create test data
bs = 2
image_base = torch.rand(bs, 3, 4, 4)  # [bs, 3, H, W] in [0, 1]
image_victim = torch.rand(bs, 3, 4, 4)  # [bs, 3, H, W] in [0, 1]

# Create encoder
image_encoder = MinimalImageEncoder(embedding_dim=8)

# Create distance function
emb_dist = minimal_emb_dist

# Create differentiable augmentation
diff_aug = MinimalDiffAug()

# Build ARGS dictionary
ARGS = {
    'image_encoder': image_encoder,
    'image_base': image_base,
    'image_victim': image_victim,
    'emb_dist': emb_dist,
    'iters': 5,
    'lr': 1 / 255,
    'eps': 8 / 255,
    'diff_aug': diff_aug,
    'resume_X_adv': None,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1
Imports
torch, torchvision, tqdm — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type tuple — a tuple output has no array shape of its own; none is stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
extract_substrings Ran
ran; our contract draft was wrong, not the code
umd-huang-lab/VLM-Poisoning/MiniGPT-4/demo_v2.py
code served (permissive licence) · get_code("5c3e6655c3b3e958")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1) and returned list, ndim 1, dtype family float; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import re

def extract_substrings(string):
    index = string.rfind('}')
    if index != -1:
        string = string[:index + 1]
    pattern = '<p>(.*?)\\}(?!<)'
    matches = re.findall(pattern, string)
    substrings = [match for match in matches]
    return substrings

# Setup code to construct valid arguments
test_string = '<p>hello}</p><p>world}more</p>end}'

ARGS = {
    'string': test_string
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1
Imports
none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type list, ndim 1, dtype family float — a list output has no array shape of its own; none is stated
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
idct_8x8_ref Ran
ran; the contract check could not run (RAISES, DEP_MISSING or TIMEOUT)
umd-huang-lab/VLM-Poisoning/DiffJPEG/decompression.py
code served (permissive licence) · get_code("effed979be6d9fe9")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned ndarray, ndim 2, dtype family float, shape (8, 8) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import numpy as np
import itertools

# Create a small 8x8 image array for testing
# The function expects a single 8x8 image (or batch of images based on docstring)
np.random.seed(0)
image_data = np.random.randn(8, 8).astype(np.float32)

ARGS = {
    "image": image_data
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
numpy — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type ndarray, ndim 2, dtype family float, shape (8, 8) — shape (8, 8) as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
is_overlapping Ran
ran; the contract check could not run (RAISES, DEP_MISSING or TIMEOUT)
umd-huang-lab/VLM-Poisoning/MiniGPT-4/demo_v2.py
code served (permissive licence) · get_code("37438be3369e6493")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1) and returned bool, ndim 0, dtype family float, shape () as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch

torch.manual_seed(0)

# Analyze the function: is_overlapping takes two rectangles
# Each rectangle is a tuple/list of 4 values: (x1, y1, x2, y2)
# representing left, top, right, bottom coordinates

# Create two rectangles that overlap
rect1 = (0, 0, 5, 5)  # Rectangle from (0,0) to (5,5)
rect2 = (3, 3, 8, 8)  # Rectangle from (3,3) to (8,8) - overlaps with rect1

ARGS = {
    "rect1": rect1,
    "rect2": rect2,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type bool, ndim 0, dtype family float, shape () — shape () as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
load_image Ran
ran, honoured the contract we drafted
umd-huang-lab/VLM-Poisoning/LLaVA/predict.py
code served (permissive licence) · get_code("9b3c1cb391672ccb")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned Image. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import tempfile
from PIL import Image
import os

# Create a temporary image file for testing
temp_dir = tempfile.mkdtemp()
temp_image_path = os.path.join(temp_dir, "test_image.jpg")

# Create a simple test image
test_image = Image.new('RGB', (100, 100), color=(73, 109, 137))
test_image.save(temp_image_path)

# Define ARGS with a local file path
ARGS = {
    'image_file': temp_image_path
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
PIL, requests — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Image — a Image output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a Image output the recorded ndim 3 and dtype family int describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
load_image Ran
ran, honoured the contract we drafted
umd-huang-lab/VLM-Poisoning/eval_poison_llava.py
code served (permissive licence) · get_code("bb945d226af806a7")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned Image. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import tempfile
from PIL import Image
import os

# Create a temporary image file for testing
temp_dir = tempfile.mkdtemp()
temp_image_path = os.path.join(temp_dir, "test_image.png")

# Create a simple RGB image and save it
test_image = Image.new('RGB', (100, 100), color=(73, 109, 137))
test_image.save(temp_image_path)

ARGS = {
    'image_file': temp_image_path
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
PIL, requests — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Image — a Image output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a Image output the recorded ndim 3 and dtype family int describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
load_image Ran
ran, honoured the contract we drafted
umd-huang-lab/VLM-Poisoning/poison_llava.py
code served (permissive licence) · get_code("1780d801d03a6325")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned Image. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import numpy as np
from PIL import Image
import io

# Create a small valid RGB image in memory
def create_test_image():
    # Create a simple 4x3 RGB image (4 pixels wide, 3 pixels tall)
    img_array = np.random.randint(0, 256, (3, 4, 3), dtype=np.uint8)
    img = Image.fromarray(img_array, mode='RGB')
    
    # Save to bytes buffer to simulate a file path
    buffer = io.BytesIO()
    img.save(buffer, format='PNG')
    buffer.seek(0)
    
    # For this setup, we'll create a temporary file-like path
    # Since we can't do actual file I/O, we'll use a workaround
    import tempfile
    import os
    temp_file = tempfile.NamedTemporaryFile(delete=False, suffix='.png')
    temp_file.write(buffer.getvalue())
    temp_file.close()
    return temp_file.name

image_path = create_test_image()

ARGS = {
    'image_path': image_path,
    'show_image': False,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
PIL, matplotlib, numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Image — a Image output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a Image output the recorded ndim 3 and dtype family int describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
load_image_tensors Ran
ran, honoured the contract we drafted
umd-huang-lab/VLM-Poisoning/poison_llava.py
code served (permissive licence) · get_code("289a615cef916907")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned tuple, ndim 5, dtype family float; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import os
import json
import tempfile
import torch
from PIL import Image
import numpy as np

torch.manual_seed(0)

# Create a temporary directory structure with the required files
temp_dir = tempfile.mkdtemp()
base_train_dir = os.path.join(temp_dir, 'base_train')
target_train_dir = os.path.join(temp_dir, 'target_train')
os.makedirs(base_train_dir)
os.makedirs(target_train_dir)

# Create cap.json with annotations
cap_data = {
    'annotations': [
        {'image_id': 0},
        {'image_id': 1}
    ]
}
with open(os.path.join(base_train_dir, 'cap.json'), 'w') as f:
    json.dump(cap_data, f)

# Create dummy images (2 pairs of base and target images)
for image_id in [0, 1]:
    # Create base image
    base_img_array = np.random.randint(0, 256, (32, 32, 3), dtype=np.uint8)
    base_img = Image.fromarray(base_img_array, 'RGB')
    base_img.save(os.path.join(base_train_dir, f'{image_id}.png'))
    
    # Create target image
    target_img_array = np.random.randint(0, 256, (32, 32, 3), dtype=np.uint8)
    target_img = Image.fromarray(target_img_array, 'RGB')
    target_img.save(os.path.join(target_train_dir, f'{image_id}.png'))

ARGS = {
    'task_data_pth': temp_dir,
    'img_size': 4
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
PIL, matplotlib, numpy, torch, torchvision — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type tuple, ndim 5, dtype family float — a tuple output has no array shape of its own; none is stated
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
quality_to_factor Ran
ran, honoured the contract we drafted
umd-huang-lab/VLM-Poisoning/DiffJPEG/utils.py
code served (permissive licence) · get_code("8ae1de364298b758")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned float, ndim 0, dtype family float, shape () as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import numpy as np

# Construct valid arguments for quality_to_factor
# The function takes a single parameter: quality (float)
# It performs arithmetic operations on it and returns a float

ARGS = {
    "quality": 75.0
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type float, ndim 0, dtype family float, shape () — shape () as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
getAug_lavisCLIP Not yet run
unverified (harvested, no recorded run)
umd-huang-lab/VLM-Poisoning/augmentation_zoo.py
code served (permissive licence) · get_code("b6264dd23f32f494")
How it ran

no invocation recorded.

rgb_to_ycbcr Not yet run
unverified (harvested, no recorded run)
umd-huang-lab/VLM-Poisoning/DiffJPEG/compression.py
code served (permissive licence) · get_code("f9b6705425c9fde9")
How it ran

no invocation recorded.

rgb_to_ycbcr_jpeg Not yet run
unverified (harvested, no recorded run)
umd-huang-lab/VLM-Poisoning/DiffJPEG/compression.py
code served (permissive licence) · get_code("b173cfe45291c442")
How it ran

no invocation recorded.

Also present in other papers' code

Of 15 samples lifted from this paper's linked repositories, 15 carry a function identity and 0 do not. Of the 15 distinct functions with an identity, 11 are also present in other papers whose code we hold (11 with at least one paper dated earlier, 2 the same month, 5 later) and 4 are only here among the 52,977 papers whose code Syntology holds (as of 2026-09-26).

Only here means: among the 52,977 papers whose code Syntology holds (51,887 with lifted samples, 1,090 through their assemblies only), as of 2026-09-26, no other paper's lifted samples or current assembly bricks contain an identical function body after normalisation.

FunctionAlso present inEarliest-datedOthers named
load_image 85 other papers
14 dated earlier, 4 the same month, 63 later, 4 same year, order not determinable
2112.05682 (2021-12)
Self-attention Does Not Need $O(n^2)$ Memory
2305.10973 (2023-05)
Drag Your GAN: Interactive Point-based Manipulation on the Generative Image Manifold
2307.10490 (2023-07)
Abusing Images and Sounds for Indirect Instruction Injection in Multi-Modal LLMs
and 82 more not named here
load_image 10 other papers
1 dated earlier, 1 the same month, 7 later, 1 same year, order not determinable
2311.06783 (2023-11)
Q-Instruct: Improving Low-level Visual Abilities for Multi-modality Foundation Models
2024.findings-acl.573 (2024 (year only))
Visual Hallucinations of Multi-modal Large Language Models
2402.14683 (2024-02)
Visual Hallucinations of Multi-modal Large Language Models
and 7 more not named here
computeIoU 6 other papers
2 dated earlier, 3 later, 1 same year, order not determinable
2310.09478 (2023-10)
MiniGPT-v2: large language model as a unified interface for vision-language multi-task learning
2312.16862 (2023-12)
TinyGPT-V: Efficient Multimodal Large Language Model via Small Backbones
2024.findings-emnlp.803 (2024 (year only))
Cross-modality Information Check for Detecting Jailbreaking in Multimodal Large Language Models
and 3 more not named here
diff_round 6 other papers
6 dated earlier
2108.09034 (2021-08)
AdvDrop: Adversarial Attack to DNNs by Dropping Information
2207.03574 (2022-07)
Demystifying the Adversarial Robustness of Random Transformation Defenses
and 3 more not named here
extract_substrings 5 other papers
3 dated earlier, 2 later
2310.09478 (2023-10)
MiniGPT-v2: large language model as a unified interface for vision-language multi-task learning
2311.15826 (2023-11)
GeoChat: Grounded Large Vision-Language Model for Remote Sensing
2312.16862 (2023-12)
TinyGPT-V: Efficient Multimodal Large Language Model via Small Backbones
and 2 more not named here
is_overlapping 5 other papers
2 dated earlier, 3 later
2310.09478 (2023-10)
MiniGPT-v2: large language model as a unified interface for vision-language multi-task learning
2312.16862 (2023-12)
TinyGPT-V: Efficient Multimodal Large Language Model via Small Backbones
2403.02076 (2024-03)
VTG-GPT: Tuning-Free Zero-Shot Video Temporal Grounding with GPT
and 2 more not named here
quality_to_factor 4 other papers
4 dated earlier
2108.09034 (2021-08)
AdvDrop: Adversarial Attack to DNNs by Dropping Information
2309.03020 (2023-09)
SEAL: A Framework for Systematic Evaluation of Real-World Super-Resolution
and 1 more not named here
chroma_subsampling 2 other papers
2 dated earlier
Wu_Robust_Image_Forgery_Detection_Over_Online_Social_Network_Shared_Images_CVPR_2022_paper (2022 (year only))
Robust Image Forgery Detection over Online Social Network Shared Images
2308.11804 (2023-08)
Adversarial Illusions in Multi-Modal Embeddings
rgb_to_ycbcr 2 other papers
2 dated earlier
Wu_Robust_Image_Forgery_Detection_Over_Online_Social_Network_Shared_Images_CVPR_2022_paper (2022 (year only))
Robust Image Forgery Detection over Online Social Network Shared Images
2308.11804 (2023-08)
Adversarial Illusions in Multi-Modal Embeddings
idct_8x8_ref 1 other paper
1 dated earlier
Wu_Robust_Image_Forgery_Detection_Over_Online_Social_Network_Shared_Images_CVPR_2022_paper (2022 (year only))
Robust Image Forgery Detection over Online Social Network Shared Images
rgb_to_ycbcr_jpeg 1 other paper
1 dated earlier
Wu_Robust_Image_Forgery_Detection_Over_Online_Social_Network_Shared_Images_CVPR_2022_paper (2022 (year only))
Robust Image Forgery Detection over Online Social Network Shared Images

Presence, not direction: an identical function body in two papers' repositories does not say who wrote it, which came from which, or that either paper's method rests on it. Matching is exact after normalisation, so a renamed or edited function is not matched. Dates are when papers appeared, not when code was written. Only .py files the harvester lifted are covered; notebooks are not read. A new-style arXiv id YYMM.NNNN dates a paper to the month 20YY-MM; otherwise Paper.year dates it to the year; otherwise it is undated. Dates are when the paper appeared, not when the code was written. Read 2026-11-22.

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

Vision-Language Models (VLMs) excel in generating textual responses from visual inputs, but their versatility raises security concerns. This study takes the first step in exposing VLMs' susceptibility to data poisoning attacks that can manipulate responses to innocuous, everyday prompts. We introduce Shadowcast, a stealthy data poisoning attack where poison samples are visually indistinguishable from benign images with matching texts. Shadowcast demonstrates effectiveness in two attack types. The first is a traditional Label Attack, tricking VLMs into misidentifying class labels, such as confusing Donald Trump for Joe Biden. The second is a novel Persuasion Attack, leveraging VLMs' text generation capabilities to craft persuasive and seemingly rational narratives for misinformation, such as portraying junk food as healthy. We show that Shadowcast effectively achieves the attacker's intentions using as few as 50 poison samples. Crucially, the poisoned samples demonstrate transferability across different VLM architectures, posing a significant concern in black-box settings. Moreover, Shadowcast remains potent under realistic conditions involving various text prompts, training data augmentation, and image compression techniques. This work reveals how poisoned VLMs can disseminate convincing yet deceptive misinformation to everyday, benign users, emphasizing the importance of data integrity for responsible VLM deployments. Our code is available at: https://github.com/umd-huang-lab/VLM-Poisoning.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2402.06659")
get_code_for_paper("2402.06659")
have("2402.06659")

The run record, dated, one paper per request, free:

curl https://syntology.ai/api/ran/2402.06659.json

A badge for a README (the split and the date, never a ratio):

[![Syntology run record](https://syntology.ai/api/ran/2402.06659.svg)](https://syntology.ai/paper/2402.06659)

Connect an agent — have() is free.