Ming-Hsuan Yang, Yibo Yang, Yunhai Tong, Bernard Ghanem, Kai Chen, Xiangtai Li, Jingbo Wang, Haobo Yuan, Lu Qi, Shilin Xu, Qingyu Shi, Yining Li
We lifted 3 functions out of this paper's own repositories and ran 2 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| xushilin1/RAP-SAM | canonical | 2 of 3 |
| Function | Status | Where it lives |
|---|---|---|
| get_rel_pos | Ran | xushilin1/RAP-SAM/ext/sam/image_encoder.py code served (permissive licence) · get_code("733d7f0bedcb74c2") |
| window_partition | Ran | xushilin1/RAP-SAM/ext/sam/image_encoder.py code served (permissive licence) · get_code("105fa08885dc36cc") |
| window_unpartition | Not yet run | xushilin1/RAP-SAM/ext/sam/image_encoder.py code served (permissive licence) · get_code("27be441cc8213e52") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Recent segmentation methods, which adopt large-scale data training and transformer architecture, aim to create one foundation model that can perform multiple tasks. However, most of these methods rely on heavy encoder and decoder frameworks, hindering their performance in real-time scenarios. To explore real-time segmentation, recent advancements primarily focus on semantic segmentation within specific environments, such as autonomous driving. However, they often overlook the generalization ability of these models across diverse scenarios. Therefore, to fill this gap, this work explores a novel real-time segmentation setting called real-time multi-purpose segmentation. It contains three fundamental sub-tasks: interactive segmentation, panoptic segmentation, and video instance segmentation. Unlike previous methods, which use a specific design for each task, we aim to use only a single end-to-end model to accomplish all these tasks in real-time. To meet real-time requirements and balance multi-task learning, we present a novel dynamic convolution-based method, Real-Time Multi-Purpose SAM (RMP-SAM). It contains an efficient encoder and an efficient decoupled adapter to perform prompt-driven decoding. Moreover, we further explore different training strategies and one new adapter design to boost co-training performance further. We benchmark several strong baselines by extending existing works to support our multi-purpose segmentation. Extensive experiments demonstrate that RMP-SAM is effective and generalizes well on proposed benchmarks and other specific semantic tasks. Our implementation of RMP-SAM achieves the optimal balance between accuracy and speed for these tasks. The code is released at https://github.com/xushilin1/RAP-SAM
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2401.10228")
get_code_for_paper("2401.10228")
have("2401.10228")
Connect an agent — have() is free.