Udo Schlegel, Mennatallah El-Assady, Carsten Lüth, Till Bungert, Lukas Klein, Paul Jäger
We lifted 11 functions out of this paper's own repositories and ran 5 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| iml-dkfz/latec | canonical | 3 of 8 |
| IML-DKFZ/latec | canonical | 2 of 3 |
| Function | Status | Where it lives |
|---|---|---|
| feature_mask | Ran | iml-dkfz/latec/src/utils/reshape_transforms.py code served (permissive licence) · get_code("aba56ac24bacff5a") |
| get_hidden_layer_eval | Ran | iml-dkfz/latec/src/utils/hidden_layer_selection.py code served (permissive licence) · get_code("959264dd199e81b7") |
| get_pylogger | Ran | iml-dkfz/latec/src/utils/pylogger.py code served (permissive licence) · get_code("aebb727eed75bd10") |
| load_evaluation_scores | Ran | IML-DKFZ/latec/src/main/main_rank.py code served (permissive licence) · get_code("68054fd89f9d291d") |
| normalize_data | Ran | IML-DKFZ/latec/src/main/main_rank.py code served (permissive licence) · get_code("61272ae2b64a89b2") |
| add_p_value_annotation | Not yet run | iml-dkfz/latec/src/utils/plot_utils.py code served (permissive licence) · get_code("f5d0af3a9faa7bc7") |
| get_hidden_layer | Not yet run | iml-dkfz/latec/src/utils/hidden_layer_selection.py code served (permissive licence) · get_code("6da1743ee500f0fb") |
| left_align_facet_plot_titles | Not yet run | iml-dkfz/latec/src/utils/plot_utils.py code served (permissive licence) · get_code("2793fcdf7c5b1c54") |
| prepare_modalities | Not yet run | IML-DKFZ/latec/src/main/main_rank.py code served (permissive licence) · get_code("c8cd26a07b4abbd3") |
| reshape_transform_2D | Not yet run | iml-dkfz/latec/src/utils/reshape_transforms.py code served (permissive licence) · get_code("361997639f735d7a") |
| reshape_transform_3D | Not yet run | iml-dkfz/latec/src/utils/reshape_transforms.py code served (permissive licence) · get_code("22c188f68d075a1a") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Explainable AI (XAI) is a rapidly growing domain with a myriad of proposed methods as well as metrics aiming to evaluate their efficacy. However, current studies are often of limited scope, examining only a handful of XAI methods and ignoring underlying design parameters for performance, such as the model architecture or the nature of input data. Moreover, they often rely on one or a few metrics and neglect thorough validation, increasing the risk of selection bias and ignoring discrepancies among metrics. These shortcomings leave practitioners confused about which method to choose for their problem. In response, we introduce LATEC, a large-scale benchmark that critically evaluates 17 prominent XAI methods using 20 distinct metrics. We systematically incorporate vital design parameters like varied architectures and diverse input modalities, resulting in 7,560 examined combinations. Through LATEC, we showcase the high risk of conflicting metrics leading to unreliable rankings and consequently propose a more robust evaluation scheme. Further, we comprehensively evaluate various XAI methods to assist practitioners in selecting appropriate methods aligning with their needs. Curiously, the emerging top-performing method, Expected Gradients, is not examined in any relevant related study. LATEC reinforces its role in future XAI research by publicly releasing all 326k saliency maps and 378k metric scores as a (meta-)evaluation dataset. The benchmark is hosted at: https://github.com/IML-DKFZ/latec.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2409.16756")
get_code_for_paper("2409.16756")
have("2409.16756")
Connect an agent — have() is free.