Nikola Simidjievski, Mateja Jamnik, Konstantin Hemker
We lifted 11 functions out of this paper's own repositories and ran 8 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| konst-int-i/healnet | canonical | 3 of 3 |
| konst-int-i/mm-lego | pwc_unofficial | 5 of 8 |
| Function | Status | Where it lives |
|---|---|---|
| Reg_Block | Ran | konst-int-i/mm-lego/mm_lego/models/baselines/mcat.py code served (permissive licence) · get_code("21c59012b6a28663") |
| SNN_Block | Ran | konst-int-i/mm-lego/mm_lego/models/baselines/mcat.py code served (permissive licence) · get_code("10482673211edd0c") |
| cache_fn | Ran | konst-int-i/healnet/healnet/models/healnet.py code served (permissive licence) · get_code("b64b66cc00d70b2f") |
| default | Ran | konst-int-i/healnet/healnet/models/healnet.py code served (permissive licence) · get_code("13404bd487501696") |
| default | Ran | konst-int-i/mm-lego/mm_lego/models/model_utils.py code served (permissive licence) · get_code("60fff7c3c400d7ff") |
| exists | Ran | konst-int-i/healnet/healnet/models/healnet.py code served (permissive licence) · get_code("5b44cb09cc8a80a0") |
| exists | Ran | konst-int-i/mm-lego/mm_lego/models/model_utils.py code served (permissive licence) · get_code("aa5486a3650902d8") |
| l1_loss | Ran | konst-int-i/mm-lego/mm_lego/models/losses.py code served (permissive licence) · get_code("97c161d1309960d6") |
| cache_fn | Not yet run | konst-int-i/mm-lego/mm_lego/models/model_utils.py code served (permissive licence) · get_code("e56401f8aa027f99") |
| get_model_parameters | Not yet run | konst-int-i/mm-lego/mm_lego/pipeline/base.py code served (permissive licence) · get_code("78f0d5d3d3b8e58a") |
| nll_loss | Not yet run | konst-int-i/mm-lego/mm_lego/models/losses.py code served (permissive licence) · get_code("8109ef4e167f0ad9") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Technological advances in medical data collection, such as high-throughput genomic sequencing and digital high-resolution histopathology, have contributed to the rising requirement for multimodal biomedical modelling, specifically for image, tabular and graph data. Most multimodal deep learning approaches use modality-specific architectures that are often trained separately and cannot capture the crucial cross-modal information that motivates the integration of different data sources. This paper presents the Hybrid Early-fusion Attention Learning Network (HEALNet) -a flexible multimodal fusion architecture, which: a) preserves modality-specific structural information, b) captures the cross-modal interactions and structural information in a shared latent space, c) can effectively handle missing modalities during training and inference, and d) enables intuitive model inspection by learning on the raw data input instead of opaque embeddings. We conduct multimodal survival analysis on Whole Slide Images and Multi-omic data on four cancer datasets from The Cancer Genome Atlas (TCGA). HEALNet achieves state-of-the-art performance compared to other end-to-end trained fusion models, substantially improving over unimodal and multimodal baselines whilst being robust in scenarios with missing modalities. The code is available at https://github.com/konst-int-i/healnet. 38th Conference on Neural Information Processing Systems (NeurIPS 2024).
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2311.09115")
get_code_for_paper("2311.09115")
have("2311.09115")
Connect an agent — have() is free.