SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2311.16491 · 2023

$Z^*$: Zero-shot Style Transfer via Attention Rearrangement

arXiv · PDF · Open in the Atlas

Code that ran

We lifted 5 functions out of this paper's own repositories and ran 3 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.

FunctionStatusWhere it lives
diffusion_step Ran HolmesShuan/Zero-shot-Style-Transfer-via-Attention-Rearrangement/ptp_utils.py
code served (permissive licence) · get_code("a042902caede448b")
latent2image Ran HolmesShuan/Zero-shot-Style-Transfer-via-Attention-Rearrangement/ptp_utils.py
code served (permissive licence) · get_code("e3c3bb87bbe9b4a7")
text_under_image Ran HolmesShuan/Zero-shot-Style-Transfer-via-Attention-Rearrangement/ptp_utils.py
code served (permissive licence) · get_code("85d5eb123674597d")
regiter_attention_editor_diffusers Not yet run HolmesShuan/Zero-shot-Style-Transfer-via-Attention-Rearrangement/zstar/zstar_utils.py
code served (permissive licence) · get_code("d321812b8f73ef4b")
regiter_attention_editor_ldm Not yet run HolmesShuan/Zero-shot-Style-Transfer-via-Attention-Rearrangement/zstar/zstar_utils.py
code served (permissive licence) · get_code("431f768e76ae86fa")

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

Despite the remarkable progress in image style transfer, formulating style in the context of art is inherently subjective and challenging. In contrast to existing learning/tuning methods, this study shows that vanilla diffusion models can directly extract style information and seamlessly integrate the generative prior into the content image without retraining. Specifically, we adopt dual denoising paths to represent content/style references in latent space and then guide the content image denoising process with style latent codes. We further reveal that the cross-attention mechanism in latent diffusion models tends to blend the content and style images, resulting in stylized outputs that deviate from the original content image. To overcome this limitation, we introduce a cross-attention rearrangement strategy. Through theoretical analysis and experiments, we demonstrate the effectiveness and superiority of the diffusion-based $\underline{Z}$ero-shot $\underline{S}$tyle $\underline{T}$ransfer via $\underline{A}$ttention $\underline{R}$earrangement, Z-STAR.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2311.16491")
get_code_for_paper("2311.16491")
have("2311.16491")

Connect an agent — have() is free.