Wenqi Shao, Ping Luo, Zhenguo Li, Enze Xie, Jungong Han, Yutao Hu, Qixiong Wang
We lifted 13 functions out of this paper's own repositories and ran 9 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| toggle1995/RIS-DMMI | canonical | 9 of 13 |
| Function | Status | Where it lives |
|---|---|---|
| window_partition | Ran | toggle1995/RIS-DMMI/lib/backbone.py pointer only (licence: NONE) · get_code("144d10b49baeb8a6") |
| find_pruneable_heads_and_indices | Ran | toggle1995/RIS-DMMI/bert/modeling_utils.py pointer only (licence: NONE) · get_code("b239247b87d4f3dc") |
| gelu | Ran | toggle1995/RIS-DMMI/bert/modeling.py pointer only (licence: NONE) · get_code("d08f324f950148de") |
| get_activation | Ran | toggle1995/RIS-DMMI/bert/modeling.py pointer only (licence: NONE) · get_code("5842ee67106321d6") |
| mish | Ran | toggle1995/RIS-DMMI/bert/modeling_bert.py pointer only (licence: NONE) · get_code("22b41155e451683b") |
| pad_if_smaller | Ran | toggle1995/RIS-DMMI/transforms.py pointer only (licence: NONE) · get_code("452475fb6da97402") |
| prune_conv1d_layer | Ran | toggle1995/RIS-DMMI/bert/modeling_utils.py pointer only (licence: NONE) · get_code("b3ee64494a3b6d38") |
| prune_linear_layer | Ran | toggle1995/RIS-DMMI/bert/modeling_utils.py pointer only (licence: NONE) · get_code("a37615738b3ddcd3") |
| window_reverse | Ran | toggle1995/RIS-DMMI/lib/backbone.py pointer only (licence: NONE) · get_code("61bf152e6a42a184") |
| dmmi_res | Not yet run | toggle1995/RIS-DMMI/lib/segmentation.py pointer only (licence: NONE) · get_code("7a437a829041ee13") |
| dmmi_swin | Not yet run | toggle1995/RIS-DMMI/lib/segmentation.py pointer only (licence: NONE) · get_code("6d20e15ac43f4f86") |
| get_assignment_map_from_checkpoint | Not yet run | toggle1995/RIS-DMMI/bert/modeling.py pointer only (licence: NONE) · get_code("50958618b65e514e") |
| load_tf_weights_in_bert | Not yet run | toggle1995/RIS-DMMI/bert/modeling_bert.py pointer only (licence: NONE) · get_code("fb9398de2966d6de") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Referring image segmentation aims to segment the target object referred by a natural language expression. However, previous methods rely on the strong assumption that one sentence must describe one target in the image, which is often not the case in real-world applications. As a result, such methods fail when the expressions refer to either no objects or multiple objects. In this paper, we address this issue from two perspectives. First, we propose a Dual Multi-Modal Interaction (DMMI) Network, which contains two decoder branches and enables information flow in two directions. In the text-to-image decoder, text embedding is utilized to query the visual feature and localize the corresponding target. Meanwhile, the image-to-text decoder is implemented to reconstruct the erased entity-phrase conditioned on the visual feature. In this way, visual features are encouraged to contain the critical semantic information about target entity, which supports the accurate segmentation in the text-to-image decoder in turn. Secondly, we collect a new challenging but realistic dataset called Ref-ZOM, which includes image-text pairs under different settings. Extensive experiments demonstrate our method achieves state-of-the-art performance on different datasets, and the Ref-ZOM-trained model performs well on various types of text inputs. Codes and datasets are available at https://github.com/toggle1995/RIS-DMMI.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2308.13853")
get_code_for_paper("2308.13853")
have("2308.13853")
Connect an agent — have() is free.