Shanghang Zhang, Renrui Zhang, Robotics, Zhenyu Wang, Yandong Guo, Kaichen Zhou, Xiaoqi Li, Jiaming Liu, Mengzhen Liu, Pengju An, Senqiao Yang, Openvla Manipllm, and 1 more
We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. We have no repository linked to it either. That is a statement about our graph, not about the paper.
A fundamental objective in robot manipulation is to enable models to comprehend visual scenes and execute actions. Although existing Vision-Language-Action (VLA) models for robots can handle a range of basic tasks, they still face challenges in two areas: (1) insufficient reasoning ability to tackle complex tasks, and (2) high computational costs for VLA model fine-tuning and inference. The recently proposed state space model (SSM) known as Mamba demonstrates promising capabilities in non-trivial sequence modeling with linear inference complexity. Inspired by this, we introduce RoboMamba, an end-to-end robotic VLA model that leverages Mamba to deliver both robotic reasoning and action capabilities, while maintaining efficient fine-tuning and inference. Specifically, we first integrate the vision encoder with Mamba, aligning visual tokens with language embedding through co-training, empowering our model with visual common sense and roboticrelated reasoning. To further equip RoboMamba with SE(3) pose prediction abilities, we explore an efficient fine-tuning strategy with a simple policy head. We find that once RoboMamba possesses sufficient reasoning capability, it can acquire manipulation skills with minimal fine-tuning parameters (0.1% of the model) and time. In experiments, RoboMamba demonstrates outstanding reasoning capabilities on general and robotic evaluation benchmarks. Meanwhile, our model showcases impressive pose prediction results in both simulation and real-world experiments, achieving inference speeds 3 times faster than existing VLA models. Our project web page: https://sites.google.com/view/robomamba-web
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2406.04339")
get_code_for_paper("2406.04339")
have("2406.04339")
The run record, dated, one paper per request, free:
curl https://syntology.ai/api/ran/2406.04339.json
A badge for a README (the split and the date, never a ratio):
[](https://syntology.ai/paper/2406.04339)
Connect an agent — have() is free.