Guillaume Jaume, Richard Chen, Faisal Mahmood, Andrew Song, Anurag Vaidya, Alexander Baras
We lifted 12 functions out of this paper's own repositories and ran 7 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| mahmoodlab/MMP | — | 7 of 12 |
| Function | Status | Where it lives |
|---|---|---|
| CoxLoss | Ran | mahmoodlab/MMP/src/mil_models/model_multimodal.py pointer only (licence: NOASSERTION) · get_code("05faa7f292aabeb5") |
| FeedForward | Ran | mahmoodlab/MMP/src/mil_models/model_multimodal.py pointer only (licence: NOASSERTION) · get_code("a206b906c4df0eba") |
| NLLSurvLoss | Ran | mahmoodlab/MMP/src/mil_models/model_multimodal.py pointer only (licence: NOASSERTION) · get_code("71951ad3b92ea5b9") |
| SNN_Block | Ran | mahmoodlab/MMP/src/mil_models/model_multimodal.py pointer only (licence: NOASSERTION) · get_code("10482673211edd0c") |
| SurvRankingLoss | Ran | mahmoodlab/MMP/src/mil_models/model_multimodal.py pointer only (licence: NOASSERTION) · get_code("95489d9d0796dea4") |
| init_per_path_model | Ran | mahmoodlab/MMP/src/mil_models/model_multimodal.py pointer only (licence: NOASSERTION) · get_code("44e13a864df9f026") |
| partial_ll_loss | Ran | mahmoodlab/MMP/src/mil_models/model_multimodal.py pointer only (licence: NOASSERTION) · get_code("4dd75c9172f12cfb") |
| MMAttention | Not yet run | mahmoodlab/MMP/src/mil_models/model_multimodal.py pointer only (licence: NOASSERTION) · get_code("721d2d0c102ab4ae") |
| MMAttentionLayer | Not yet run | mahmoodlab/MMP/src/mil_models/model_multimodal.py pointer only (licence: NOASSERTION) · get_code("323ddf11c2d03fca") |
| SurvPath | Not yet run | mahmoodlab/MMP/src/mil_models/model_multimodal.py pointer only (licence: NOASSERTION) · get_code("b3e9ddc34af4aad1") |
| nll_loss | Not yet run | mahmoodlab/MMP/src/mil_models/model_multimodal.py pointer only (licence: NOASSERTION) · get_code("15821d5a95c321e7") |
| process_surv | Not yet run | mahmoodlab/MMP/src/mil_models/model_multimodal.py pointer only (licence: NOASSERTION) · get_code("64e4caddca1b2fb3") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Multimodal survival methods combining gigapixel histology whole-slide images (WSIs) and transcriptomic profiles are particularly promising for patient prognostication and stratification. Current approaches involve tokenizing the WSIs into smaller patches (> 10 4 patches) and transcriptomics into gene groups, which are then integrated using a Transformer for predicting outcomes. However, this process generates many tokens, which leads to high memory requirements for computing attention and complicates post-hoc interpretability analyses. Instead, we hypothesize that we can: (1) effectively summarize the morphological content of a WSI by condensing its constituting tokens using morphological prototypes, achieving more than 300× compression; and (2) accurately characterize cellular functions by encoding the transcriptomic profile with biological pathway prototypes, all in an unsupervised fashion. The resulting multimodal tokens are then processed by a fusion network, either with a Transformer or an optimal transport cross-alignment, which now operates with a small and fixed number of tokens without approximations. Extensive evaluation on six cancer types shows that our framework outperforms state-of-the-art methods with much less computation while unlocking new interpretability analyses. The code is available at https: //github.com/mahmoodlab/MMP.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2407.00224")
get_code_for_paper("2407.00224")
have("2407.00224")
Connect an agent — have() is free.