Honglak Lee, Seung Kim, Jonghyun Choi, Dongyeop Kang, Daechul Ahn, Daneul Kim, Gwangmo Song
We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Story visualization (SV) is a challenging text-to-image generation task for the difficulty of not only rendering visual details from the text descriptions but also encoding a longterm context across multiple sentences. While prior efforts mostly focus on generating a semantically relevant image for each sentence, encoding a context spread across the given paragraph to generate contextually convincing images (e.g., with a correct character or with a proper background of the scene) remains a challenge. To this end, we propose a novel memory architecture for the Bi-directional Transformer framework with an online text augmentation that generates multiple pseudo-descriptions as supplementary supervision during training for better generalization to the language variation at inference. In extensive experiments on the two popular SV benchmarks, i.e., the Pororo-SV and Flintstones-SV, the proposed method significantly outperforms the state of the arts in various metrics including FID, character F1, frame accuracy, BLEU-2/3, and R-precision with similar or less computational complexity.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2308.07575")
get_code_for_paper("2308.07575")
have("2308.07575")
The run record, dated, one paper per request, free:
curl https://syntology.ai/api/ran/2308.07575.json
A badge for a README (the split and the date, never a ratio):
[](https://syntology.ai/paper/2308.07575)
Connect an agent — have() is free.