SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2403.02817 · 2024

Here Comes The AI Worm: Unleashing Zero-click Worms that Target GenAI-Powered Applications

arXiv · PDF · Open in the Atlas

Code that ran

We lifted 8 functions out of this paper's own repositories and ran 2 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.

RepositoryRoleRan
stavc/compromptmized canonical 2 of 8
FunctionStatusWhere it lives
receive_complete_data Ran stavc/compromptmized/Legacy_Arxiv_V1/FlowSteering/ApplicationCode/EmailServer/EmailServer.py
pointer only (licence: NONE) · get_code("3e0c20655c425cf6")
receive_complete_data Ran stavc/compromptmized/Legacy_Arxiv_V1/FlowSteering/ApplicationCode/EndUserCode/EndUserClient.py
pointer only (licence: NONE) · get_code("c3fcbd96976218e7")
load_model Not yet run stavc/compromptmized/Legacy_Arxiv_V1/FlowSteering/llava/serve/model_worker.py
pointer only (licence: NONE) · get_code("a2445eba62e3b93d")
parse_email_data Not yet run stavc/compromptmized/Legacy_Arxiv_V1/FlowSteering/ApplicationCode/EndUserCode/EndUserClient.py
pointer only (licence: NONE) · get_code("ec85aba0605e5453")
pretty_print_semaphore Not yet run stavc/compromptmized/Legacy_Arxiv_V1/FlowSteering/llava/utils.py
pointer only (licence: NONE) · get_code("37899f22fb191b37")
send_Email Not yet run stavc/compromptmized/Legacy_Arxiv_V1/FlowSteering/ApplicationCode/EndUserCode/AttackerClient.py
pointer only (licence: NONE) · get_code("34962d308ed86b96")
send_Email Not yet run stavc/compromptmized/Legacy_Arxiv_V1/FlowSteering/ApplicationCode/EndUserCode/EndUserClient.py
pointer only (licence: NONE) · get_code("72ab0ce247e7433d")
violates_moderation Not yet run stavc/compromptmized/Legacy_Arxiv_V1/FlowSteering/llava/utils.py
pointer only (licence: NONE) · get_code("f9939a84b9a65279")

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

In this paper, we show that when the communication between GenAI-powered applications relies on RAG-based inference, an attacker can initiate a computer worm-like chain reaction that we call Morris-II. This is done by crafting an adversarial self-replicating prompt that triggers a cascade of indirect prompt injections within the ecosystem and forces each affected application to perform malicious actions and compromise the RAG of additional applications. We evaluate the performance of the worm in creating a chain of confidential user data extraction within a GenAI ecosystem of GenAI-powered email assistants and analyze how the performance of the worm is affected by the size of the context, the adversarial self-replicating prompt used, the type and size of the embedding algorithm employed, and the number of hops in the propagation. Finally, we introduce the Virtual Donkey, a guardrail intended to detect and prevent the propagation of Morris-II with minimal latency, high accuracy, and a low false-positive rate. We evaluate the guardrail's performance and show that it yields a perfect true-positive rate of 1.0 with a false-positive rate of 0.015, and is robust against out-of-distribution worms, consisting of unseen jailbreaking commands, a different email dataset, and various worm usecases.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2403.02817")
get_code_for_paper("2403.02817")
have("2403.02817")

Connect an agent — have() is free.