Dongdong Chen, Lu Yuan, Nenghai Yu, Xiyang Dai, Yinpeng Chen, Mengchen Liu, Qi Chu, Encoder Decoder, Zhentao Tan, Qiankun Liu, Softmax Linear, መ 𝐟
We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Transformers have achieved great success in pluralistic image inpainting recently. However, we find existing transformer based solutions regard each pixel as a token, thus suffer from information loss issue from two aspects: 1) They downsample the input image into much lower resolutions for efficiency consideration, incurring information loss and extra misalignment for the boundaries of masked regions. 2) They quantize 256 3 RGB pixels to a small number (such as 512) of quantized pixels. The indices of quantized pixels are used as tokens for the inputs and prediction targets of transformer. Although an extra CNN network is used to upsample and refine the low-resolution results, it is difficult to retrieve the lost information back. To keep input information as much as possible, we propose a new transformer based framework "PUT". Specifically, to avoid input downsampling while maintaining the computation efficiency, we design a patch-based auto-encoder P-VQVAE, where the encoder converts the masked image into non-overlapped patch tokens and the decoder recovers the masked regions from the inpainted tokens while keeping the unmasked regions unchanged. To eliminate the information loss caused by quantization, an Un-Quantized Transformer (UQ-Transformer) is applied, which directly takes the features from P-VQVAE encoder as input without quantization and regards the quantized tokens only as prediction targets. Extensive experiments show that PUT greatly outperforms state-of-the-art methods on image fidelity, especially for large masked regions and complex large-scale datasets. × 𝑛 ′ Linear
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2205.05076")
get_code_for_paper("2205.05076")
have("2205.05076")
The run record, dated, one paper per request, free:
curl https://syntology.ai/api/ran/2205.05076.json
A badge for a README (the split and the date, never a ratio):
[](https://syntology.ai/paper/2205.05076)
Connect an agent — have() is free.