Zhengkai Jiang, Yunsheng Wu, Jiangning Zhang, Chengjie Wang, Kai Wu, Yabiao Wang, Qingdong He, Jinlong Peng, Ji Xiaozhong, Mingang Chen
We lifted 3 functions out of this paper's own repositories and ran 1 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| hithqd/unim-ov3d | — | 1 of 3 |
| Function | Status | Where it lives |
|---|---|---|
| SparseUNetTextSeg | Ran | hithqd/unim-ov3d/pcseg/models/vision_networks/sparseunet_textseg.py pointer only (licence: NONE) · get_code("83afeeacb9f1ca25") |
| ModelTemplate | Not yet run | hithqd/unim-ov3d/pcseg/models/vision_networks/sparseunet_textseg.py pointer only (licence: NONE) · get_code("c56bbe1cc3df19fa") |
| find_all_spconv_keys | Not yet run | hithqd/unim-ov3d/pcseg/models/vision_networks/sparseunet_textseg.py pointer only (licence: NONE) · get_code("4736609cee75702a") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
3D open-vocabulary scene understanding aims to recognize arbitrary novel categories beyond the base label space. However, existing works not only fail to fully utilize all the available modal information in the 3D domain but also lack sufficient granularity in representing the features of each modality. In this paper, we propose a unified multimodal 3D open-vocabulary scene understanding network, namely UniM-OV3D, which aligns point clouds with image, language and depth. To better integrate global and local features of the point clouds, we design a hierarchical point cloud feature extraction module that learns comprehensive finegrained feature representations. Further, to facilitate the learning of coarse-to-fine point-semantic representations from captions, we propose the utilization of hierarchical 3D caption pairs, capitalizing on geometric constraints across various viewpoints of 3D scenes. Extensive experimental results demonstrate the effectiveness and superiority of our method in open-vocabulary semantic and instance segmentation, which achieves state-of-theart performance on both indoor and outdoor benchmarks such as ScanNet, ScanNet200, S3IDS and nuScenes. Code is available at https:// github.com/ hithqd/ UniM-OV3D.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2401.11395")
get_code_for_paper("2401.11395")
have("2401.11395")
Connect an agent — have() is free.