Chen Qian, Wayne Wu, Hang Zhou, Limin Wang, Zhaoyang Liu, Haoyue Cheng
We lifted 6 functions out of this paper's own repositories and ran 5 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| mengyuanchen21/cvpr2023-cmpae | — | 3 of 3 |
| MCG-NJU/JoMoLD | — | 2 of 3 |
| Function | Status | Where it lives |
|---|---|---|
| Encoder | Ran | MCG-NJU/JoMoLD/nets/net_audiovisual.py pointer only (licence: NONE) · get_code("09c6cee14764da2c") |
| HANLayer | Ran | MCG-NJU/JoMoLD/nets/net_audiovisual.py pointer only (licence: NONE) · get_code("dc4267bfaf17ac28") |
| MMIL_Net | Ran | mengyuanchen21/cvpr2023-cmpae/nets/net_audiovisual.py code served (permissive licence) · get_code("4913d3d548ebcc79") |
| exp_evidence | Ran | mengyuanchen21/cvpr2023-cmpae/nets/net_audiovisual.py code served (permissive licence) · get_code("73ab74fee9265b2c") |
| get_p_and_u_from_logit | Ran | mengyuanchen21/cvpr2023-cmpae/nets/net_audiovisual.py code served (permissive licence) · get_code("ac4ccd836b0b4d36") |
| MMIL_Net | Not yet run | MCG-NJU/JoMoLD/nets/net_audiovisual.py pointer only (licence: NONE) · get_code("e7739be4fe052b6e") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
This paper focuses on the weakly-supervised audio-visual video parsing task, which aims to recognize all events belonging to each modality and localize their temporal boundaries. This task is challenging because only overall labels indicating the video events are provided for training. However, an event might be labeled but not appear in one of the modalities, which results in a modality-specific noisy label problem. In this work, we propose a training strategy to identify and remove modality-specific noisy labels dynamically. It is motivated by two key observations: 1) networks tend to learn clean samples first; and 2) a labeled event would appear in at least one modality. Specifically, we sort the losses of all instances within a mini-batch individually in each modality, and then select noisy samples according to the relationships between intra-modal and inter-modal losses. Besides, we also propose a simple but valid noise ratio estimation method by calculating the proportion of instances whose confidence is below a preset threshold. Our method makes large improvements over the previous state of the arts (e.g., from 60.0% to 63.8% in segment-level visual metric), which demonstrates the effectiveness of our approach. Code and trained models are publicly available at https://github.com/MCG-NJU/JoMoLD.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2204.11573")
get_code_for_paper("2204.11573")
have("2204.11573")
Connect an agent — have() is free.