Dong Liu, Kaidong Zhang, Jingjing Fu
We lifted 7 functions out of this paper's own repositories and ran 6 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| hitachinsk/FGT | canonical | 6 of 7 |
| Function | Status | Where it lives |
|---|---|---|
| calculate_ssim | Ran | hitachinsk/FGT/FGT/metrics/ssim.py code served (permissive licence) · get_code("89491f4cebb038b3") |
| parse | Ran | hitachinsk/FGT/FGT/parse.py code served (permissive licence) · get_code("8fe8568379d37392") |
| path_correction | Ran | hitachinsk/FGT/FGT/parse.py code served (permissive licence) · get_code("71518b3416044007") |
| psnr | Ran | hitachinsk/FGT/FGT/metrics/psnr.py code served (permissive licence) · get_code("88b5e0ba332f376c") |
| ssim | Ran | hitachinsk/FGT/FGT/metrics/ssim.py code served (permissive licence) · get_code("09eb612ec195fe9b") |
| val_path_correction | Ran | hitachinsk/FGT/FGT/parse.py code served (permissive licence) · get_code("f56519037bd01fad") |
| spectral_norm | Not yet run | hitachinsk/FGT/FGT/models/temporal_patch_gan.py code served (permissive licence) · get_code("b189ed0149b76880") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
We propose a flow-guided transformer, which innovatively leverage the motion discrepancy exposed by optical flows to instruct the attention retrieval in transformer for high fidelity video inpainting. More specially, we design a novel flow completion network to complete the corrupted flows by exploiting the relevant flow features in a local temporal window. With the completed flows, we propagate the content across video frames, and adopt the flow-guided transformer to synthesize the rest corrupted regions. We decouple transformers along temporal and spatial dimension, so that we can easily integrate the locally relevant completed flows to instruct spatial attention only. Furthermore, we design a flow-reweight module to precisely control the impact of completed flows on each spatial transformer. For the sake of efficiency, we introduce window partition strategy to both spatial and temporal transformers. Especially in spatial transformer, we design a dual perspective spatial MHSA, which integrates the global tokens to the window-based attention. Extensive experiments demonstrate the effectiveness of the proposed method qualitatively and quantitatively. Codes are available at https://github.com/hitachinsk/FGT.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2208.06768")
get_code_for_paper("2208.06768")
have("2208.06768")
Connect an agent — have() is free.