SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2506.01853 · 2025

ShapeLLM-Omni: A Native Multimodal LLM for 3D Generation and Understanding

arXiv · PDF · Open in the Atlas

Code that ran

We lifted 8 functions out of this paper's own repositories and ran 4 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.

RepositoryRoleRan
jamesyjl/shapellm-omni canonical 4 of 8
FunctionStatusWhere it lives
get_first_directory Ran jamesyjl/shapellm-omni/dataset_toolkits/build_metadata.py
code served (permissive licence) · get_code("29323106f1b1a408")
is_video_file Ran jamesyjl/shapellm-omni/app_old.py
code served (permissive licence) · get_code("81dcd07d6d8fcab9")
save_ply_from_array Ran jamesyjl/shapellm-omni/app_old.py
code served (permissive licence) · get_code("d8e8d96d698f1fa9")
token_to_mesh Ran jamesyjl/shapellm-omni/app_old.py
code served (permissive licence) · get_code("697215ba5d486775")
get_file_hash Not yet run jamesyjl/shapellm-omni/dataset_toolkits/utils.py
code served (permissive licence) · get_code("a691824c3709ab59")
halton_sequence Not yet run jamesyjl/shapellm-omni/dataset_toolkits/utils.py
code served (permissive licence) · get_code("2252a415d2271734")
need_process Not yet run jamesyjl/shapellm-omni/dataset_toolkits/build_metadata.py
code served (permissive licence) · get_code("ae3ac644f6a269e7")
radical_inverse Not yet run jamesyjl/shapellm-omni/dataset_toolkits/utils.py
code served (permissive licence) · get_code("df4d5ce8269a55b2")

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

Recently, the powerful text-to-image capabilities of ChatGPT-4o have led to growing appreciation for native multimodal large language models. However, its multimodal capabilities remain confined to images and text. Yet beyond images, the ability to understand and generate 3D content is equally crucial. To address this gap, we propose ShapeLLM-Omni-a native 3D large language model capable of understanding and generating 3D assets and text in any sequence. First, we train a 3D vector-quantized variational autoencoder (VQVAE), which maps 3D objects into a discrete latent space to achieve efficient and accurate shape representation and reconstruction. Building upon the 3D-aware discrete tokens, we innovatively construct a large-scale continuous training dataset named 3D-Alpaca, encompassing generation, comprehension, and editing, thus providing rich resources for future research and training. Finally, by performing instruction-based training of the Qwen-2.5-vl-7B-Instruct model on the 3D-Alpaca dataset. Our work provides an effective attempt at extending multimodal models with basic 3D capabilities, which contributes to future research in 3D-native AI. Project page: https://github.com/JAMESYJL/ShapeLLM-Omni

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2506.01853")
get_code_for_paper("2506.01853")
have("2506.01853")

Connect an agent — have() is free.