SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2308.14686 · 2023

360-Degree Panorama Generation from Few Unregistered NFoV Images

arXiv · PDF · Open in the Atlas

Code that ran

We lifted 6 functions out of this paper's own repositories and ran 3 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.

RepositoryRoleRan
shanemankiw/panodiff canonical 3 of 6
FunctionStatusWhere it lives
default_loader Ran shanemankiw/panodiff/RelativeRotation/dataset.py
pointer only (licence: NONE) · get_code("7bf07d5f59aae36d")
load_rgb Ran shanemankiw/panodiff/process_raw_BLIP.py
pointer only (licence: NONE) · get_code("d6e31e4288494a9b")
pil_loader Ran shanemankiw/panodiff/RelativeRotation/dataset.py
pointer only (licence: NONE) · get_code("f321f54723433661")
accimage_loader Not yet run shanemankiw/panodiff/RelativeRotation/dataset.py
pointer only (licence: NONE) · get_code("404fb2b2daa1ae78")
get_state_dict Not yet run shanemankiw/panodiff/cldm/model.py
pointer only (licence: NONE) · get_code("cfc17707f35e7ec4")
load_state_dict Not yet run shanemankiw/panodiff/cldm/model.py
pointer only (licence: NONE) · get_code("df733a879693145d")

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

360$^\circ$ panoramas are extensively utilized as environmental light sources in computer graphics. However, capturing a 360$^\circ$ $\times$ 180$^\circ$ panorama poses challenges due to the necessity of specialized and costly equipment, and additional human resources. Prior studies develop various learning-based generative methods to synthesize panoramas from a single Narrow Field-of-View (NFoV) image, but they are limited in alterable input patterns, generation quality, and controllability. To address these issues, we propose a novel pipeline called PanoDiff, which efficiently generates complete 360$^\circ$ panoramas using one or more unregistered NFoV images captured from arbitrary angles. Our approach has two primary components to overcome the limitations. Firstly, a two-stage angle prediction module to handle various numbers of NFoV inputs. Secondly, a novel latent diffusion-based panorama generation model uses incomplete panorama and text prompts as control signals and utilizes several geometric augmentation schemes to ensure geometric properties in generated panoramas. Experiments show that PanoDiff achieves state-of-the-art panoramic generation quality and high controllability, making it suitable for applications such as content editing.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2308.14686")
get_code_for_paper("2308.14686")
have("2308.14686")

Connect an agent — have() is free.