Jingyang Zhang, Yao Yao, Tian Fang, Yanghai Tsin, Jingbo Liu, Yihang Qu, David Mckinnon, Long Quan
We lifted 3 functions out of this paper's own repositories and ran 3 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| apple/ml-neilf | — | 3 of 3 |
| Function | Status | Where it lives |
|---|---|---|
| NeILFPBR | Ran | apple/ml-neilf/code/model/neilf_brdf.py pointer only (licence: NOASSERTION) · get_code("50ad47f06f50302e") |
| fibonacci_sphere_sampling | Ran | apple/ml-neilf/code/model/neilf_brdf.py pointer only (licence: NOASSERTION) · get_code("523ef1026f996d8c") |
| rotation_between_vectors | Ran | apple/ml-neilf/code/model/neilf_brdf.py pointer only (licence: NOASSERTION) · get_code("3abab3042b52ceae") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
We present a differentiable rendering framework for material and lighting estimation from multi-view images and a reconstructed geometry. In the framework, we represent scene lightings as the Neural Incident Light Field (NeILF) and material properties as the surface BRDF modelled by multi-layer perceptrons. Compared with recent approaches that approximate scene lightings as the 2D environment map, NeILF is a fully 5D light field that is capable of modelling illuminations of any static scenes. In addition, occlusions and indirect lights can be handled naturally by the NeILF representation without requiring multiple bounces of ray tracing, making it possible to estimate material properties even for scenes with complex lightings and geometries. We also propose a smoothness regularization and a Lambertian assumption to reduce the materiallighting ambiguity during the optimization. Our method strictly follows the physically-based rendering equation, and jointly optimizes material and lighting through the differentiable rendering process. We have intensively evaluated the proposed method on our in-house synthetic dataset, the DTU MVS dataset, and real-world BlendedMVS scenes. Our method outperforms previous methods by a significant margin in terms of novel view rendering quality, setting a new state-of-the-art for image-based material and lighting estimation.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2203.07182")
get_code_for_paper("2203.07182")
have("2203.07182")
Connect an agent — have() is free.