Latent Guard: a Safety Framework for Text-to-image Generation

Philip Torr, Qifeng Chen, Ashkan Khakzar, Runtao Liu, Jindong Gu, Fabio Pizzati

arXiv · PDF · Open in the Atlas

Code that ran

We lifted 1 functions out of this paper's own repositories and ran 1 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.

Repositories linked to this paper

• rt219/latentguard — marked official in Papers with Code; named in the paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

With the ability to generate high-quality images, text-toimage (T2I) models can be exploited for creating inappropriate content. To prevent misuse, existing safety measures are either based on text blacklists, easily circumvented, or harmful content classification, using large datasets for training and offering low flexibility. Here, we propose Latent Guard, a framework designed to improve safety measures in text-to-image generation. Inspired by blacklist-based approaches, Latent Guard learns a latent space on top of the T2I model's text encoder, where we check the presence of harmful concepts in the input text embeddings. Our framework is composed of a data generation pipeline specific to the task using large language models, ad-hoc architectural components, and a contrastive learning strategy to benefit from the generated data. Our method is evaluated on three datasets and against four baselines. Warning: This paper contains potentially offensive text and images.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2404.08031")
get_code_for_paper("2404.08031")
have("2404.08031")

Connect an agent — have() is free.