Ziwei Liu, Rui Qian, ๐ก, Lequan Yu, Xian Liu, Lingting Zhu, Xuanyu Liu, ๐ ๐ ( ๐ก-
arXiv ยท PDF ยท Open in the Atlas
We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Animating virtual avatars to make co-speech gestures facilitates various applications in human-machine interaction. The existing methods mainly rely on generative adversarial networks (GANs), which typically suffer from notorious mode collapse and unstable training, thus making it difficult to learn accurate audio-gesture joint distributions. In this work, we propose a novel diffusion-based framework, named Diffusion Co-Speech Gesture (DiffGesture), to effectively capture the cross-modal audio-to-gesture associations and preserve temporal coherence for high-fidelity audio-driven co-speech gesture generation. Specifically, we first establish the diffusion-conditional generation process on clips of skeleton sequences and audio to enable the whole framework. Then, a novel Diffusion Audio-Gesture Transformer is devised to better attend to the information from multiple modalities and model the long-term temporal dependency. Moreover, to eliminate temporal inconsistency, we propose an effective Diffusion Gesture Stabilizer with an annealed noise sampling strategy. Benefiting from the architectural advantages of diffusion models, we further incorporate implicit classifier-free guidance to trade off between diversity and gesture quality. Extensive experiments demonstrate that DiffGesture achieves state-of-theart performance, which renders coherent gestures with better mode coverage and stronger audio correlations. Code is available at https://github.com/Advocate99/DiffGesture.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2303.09119")
get_code_for_paper("2303.09119")
have("2303.09119")
The run record, dated, one paper per request, free:
curl https://syntology.ai/api/ran/2303.09119.json
A badge for a README (the split and the date, never a ratio):
[](https://syntology.ai/paper/2303.09119)
Connect an agent โ have() is free.