Honglak Lee, Han Zhang, Ting Chen, Kaist, Seunghoon Hong, Jing Koh, Thomas Huang, Wonkwang Lee, Whie Jung, Hyungsuk Yoon
We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Learning to predict the long-term future of video frames is notoriously challenging due to inherent ambiguities in the distant future and dramatic amplifications of prediction error through time. Despite the recent advances in the literature, existing approaches are limited to moderately short-term prediction (less than a few seconds), while extrapolating it to a longer future quickly leads to destruction in structure and content. In this work, we revisit hierarchical models in video prediction. Our method predicts future frames by first estimating a sequence of semantic structures and subsequently translating the structures to pixels by video-to-video translation. Despite the simplicity, we show that modeling structures and their dynamics in the discrete semantic structure space with a stochastic recurrent estimator leads to surprisingly successful long-term prediction. We evaluate our method on three challenging datasets involving car driving and human dancing, and demonstrate that it can generate complicated scene structures and motions over a very long time horizon (i.e., thousands frames), setting a new standard of video prediction with orders of magnitude longer prediction time than existing approaches. Full videos and codes are available at https://1konny.github.io/HVP/.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2104.06697")
get_code_for_paper("2104.06697")
have("2104.06697")
The run record, dated, one paper per request, free:
curl https://syntology.ai/api/ran/2104.06697.json
A badge for a README (the split and the date, never a ratio):
[](https://syntology.ai/paper/2104.06697)
Connect an agent — have() is free.