We lifted 17 functions out of this paper's own repositories and ran 9 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| OpenRobotLab/Grounded_3D-LLM | canonical | 9 of 17 |
| Function | Status | Where it lives |
|---|---|---|
| repeat_kv | Ran | OpenRobotLab/Grounded_3D-LLM/models/LLM/modeling_llama.py pointer only (licence: NONE) · get_code("30d7eec482ebf6b1") |
| apply_rotary_pos_emb | Ran | OpenRobotLab/Grounded_3D-LLM/models/LLM/modeling_llama.py pointer only (licence: NONE) · get_code("d61c483a3c2b3156") |
| extract_decoder_hidden_states | Ran | OpenRobotLab/Grounded_3D-LLM/models/LLM/llama_utils.py pointer only (licence: NONE) · get_code("af45d6b0d475f902") |
| extract_numbers_and_string | Ran | OpenRobotLab/Grounded_3D-LLM/models/metrics/evaluate_LLM.py pointer only (licence: NONE) · get_code("fe938788e6c1b91e") |
| get_batch_aabb_pair_ious | Ran | OpenRobotLab/Grounded_3D-LLM/models/metrics/evaluate_LLM.py pointer only (licence: NONE) · get_code("982eff1f79a70a95") |
| get_loss_for_each_type | Ran | OpenRobotLab/Grounded_3D-LLM/models/LLM/llama_utils.py pointer only (licence: NONE) · get_code("98cb6f5c94e2ed6a") |
| rotate_half | Ran | OpenRobotLab/Grounded_3D-LLM/models/LLM/modeling_llama.py pointer only (licence: NONE) · get_code("b99eea6376d1e212") |
| simple_ratio | Ran | OpenRobotLab/Grounded_3D-LLM/models/LLM/scanqa_score.py pointer only (licence: NONE) · get_code("d8f1ec58fa91ba5f") |
| smooth_l1 | Ran | OpenRobotLab/Grounded_3D-LLM/models/matcher.py pointer only (licence: NONE) · get_code("d4d10e9d1f354600") |
| batch_dice_loss | Not yet run | OpenRobotLab/Grounded_3D-LLM/models/matcher.py pointer only (licence: NONE) · get_code("bc2cb481a75c370d") |
| binary_focal_loss_cost | Not yet run | OpenRobotLab/Grounded_3D-LLM/models/matcher.py pointer only (licence: NONE) · get_code("4854fff95b83b510") |
| calculate_uncertainty | Not yet run | OpenRobotLab/Grounded_3D-LLM/models/criterion.py pointer only (licence: NONE) · get_code("2dcb8123d89bb1ff") |
| dice_loss | Not yet run | OpenRobotLab/Grounded_3D-LLM/models/criterion.py pointer only (licence: NONE) · get_code("89f75e54ff128be0") |
| eval_llm_iou_score | Not yet run | OpenRobotLab/Grounded_3D-LLM/models/metrics/evaluate_LLM.py pointer only (licence: NONE) · get_code("f16e9fe96aedf28e") |
| get_lemma | Not yet run | OpenRobotLab/Grounded_3D-LLM/models/LLM/scanqa_score.py pointer only (licence: NONE) · get_code("a1ff41c753bffe5d") |
| sigmoid_ce_loss | Not yet run | OpenRobotLab/Grounded_3D-LLM/models/criterion.py pointer only (licence: NONE) · get_code("d0c61e8dba511aa3") |
| tokens_unigram_f_value | Not yet run | OpenRobotLab/Grounded_3D-LLM/models/LLM/scanqa_score.py pointer only (licence: NONE) · get_code("115badbfb8522aa6") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Prior studies on 3D scene understanding have primarily developed specialized models for specific tasks or required task-specific fine-tuning. In this study, we propose Grounded 3D-LLM, which explores the potential of 3D large multi-modal models (3D LMMs) to consolidate various 3D vision tasks within a unified generative framework. The model uses scene referent tokens as special noun phrases to reference 3D scenes, enabling it to handle sequences that interleave 3D and textual data. Per-task instruction-following templates are employed to ensure natural and diversity in translating 3D vision tasks into language formats. To facilitate the use of referent tokens in subsequent language modeling, we provide a large-scale, automatically curated grounded scene-text dataset with over 1 million phrase-to-region correspondences and introduce Contrastive Language-Scene Pre-training (CLASP) to perform phrase-level scene-text alignment using this data. Our comprehensive evaluation covers open-ended tasks like dense captioning and 3D question answering, alongside close-ended tasks such as object detection and language grounding. Experiments across multiple 3D benchmarks reveal the leading performance and the broad applicability of Grounded 3D-LLM. Code and datasets are available at the https://groundedscenellm.github.io/grounded_3d-llm.github.io.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2405.10370")
get_code_for_paper("2405.10370")
have("2405.10370")
Connect an agent — have() is free.