We lifted 18 functions out of this paper's own repositories and ran 12 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| tangli-udel/DEAL | canonical | 12 of 18 |
| Function | Status | Where it lives |
|---|---|---|
| aggregate_similarity | Ran | tangli-udel/DEAL/load.py code served (permissive licence) · get_code("3936ee935ec8fce1") |
| basic_clean | Ran | tangli-udel/DEAL/CLIP/clip/simple_tokenizer.py code served (permissive licence) · get_code("98f385d847636a3e") |
| batch_gradCAM | Ran | tangli-udel/DEAL/explainer.py code served (permissive licence) · get_code("0173b09907880333") |
| get_pairs | Ran | tangli-udel/DEAL/CLIP/clip/simple_tokenizer.py code served (permissive licence) · get_code("d919ae32e5e4e616") |
| gradCAM | Ran | tangli-udel/DEAL/explainer.py code served (permissive licence) · get_code("0ec4cd6b1e544f4c") |
| load_json | Ran | tangli-udel/DEAL/loading_helpers.py code served (permissive licence) · get_code("1069918b276d2855") |
| make_descriptor_sentence | Ran | tangli-udel/DEAL/loading_helpers.py code served (permissive licence) · get_code("6775b1940ad09da2") |
| multi_head_attention_forward | Ran | tangli-udel/DEAL/CLIP/clip/auxilary.py code served (permissive licence) · get_code("33c1a0c844a11c56") |
| normalize_heatmap | Ran | tangli-udel/DEAL/loss.py code served (permissive licence) · get_code("e9b027ab862daf7d") |
| stringtolist | Ran | tangli-udel/DEAL/descriptor_strings.py code served (permissive licence) · get_code("46062947cf3ab0e6") |
| whitespace_clean | Ran | tangli-udel/DEAL/CLIP/clip/simple_tokenizer.py code served (permissive licence) · get_code("9542161e9640b858") |
| wordify | Ran | tangli-udel/DEAL/loading_helpers.py code served (permissive licence) · get_code("1812ba359126089a") |
| build_model | Not yet run | tangli-udel/DEAL/CLIP/clip/model.py code served (permissive licence) · get_code("7ecaa1ab867d461d") |
| interpret | Not yet run | tangli-udel/DEAL/explainer.py code served (permissive licence) · get_code("8bd4fcd6be04a6f6") |
| interpret | Not yet run | tangli-udel/DEAL/CLIP/example.py code served (permissive licence) · get_code("348977282f211827") |
| load | Not yet run | tangli-udel/DEAL/CLIP/clip/clip.py code served (permissive licence) · get_code("f764db9d0498c5c4") |
| mod_stringtolist | Not yet run | tangli-udel/DEAL/descriptor_strings.py code served (permissive licence) · get_code("0eac6e65494da061") |
| stringtolist_opt | Not yet run | tangli-udel/DEAL/descriptor_strings.py code served (permissive licence) · get_code("1fa486482cc060ee") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Large pre-trained Vision-Language Models (VLMs) have become ubiquitous foundational components of other models and downstream tasks. Although powerful, our empirical results reveal that such models might not be able to identify fine-grained concepts. Specifically, the explanations of VLMs with respect to fine-grained concepts are entangled and mislocalized. To address this issue, we propose to DisEntAngle and Localize (DEAL) the concept-level explanations for VLMs without human annotations. The key idea is encouraging the concept-level explanations to be distinct while maintaining consistency with category-level explanations. We conduct extensive experiments and ablation studies on a wide range of benchmark datasets and vision-language models. Our empirical results demonstrate that the proposed method significantly improves the concept-level explanations of the model in terms of disentanglability and localizability. Surprisingly, the improved explainability alleviates the model's reliance on spurious correlations, which further benefits the prediction accuracy.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2407.14412")
get_code_for_paper("2407.14412")
have("2407.14412")
Connect an agent — have() is free.