SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2403.05396 · 2024

HistGen: Histopathology Report Generation via Local-Global Feature Encoding and Cross-modal Context Interaction

arXiv · PDF · Open in the Atlas

Code that ran

We lifted 6 functions out of this paper's own repositories and ran 6 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.

RepositoryRoleRan
dddavid4real/HistGen canonical 6 of 6
FunctionStatusWhere it lives
attention Ran dddavid4real/HistGen/models/PlainTransformer_Modules.py
code served (permissive licence) · get_code("269003e7c21d565d")
clones Ran dddavid4real/HistGen/models/PlainTransformer_Modules.py
code served (permissive licence) · get_code("a3722169bbc81569")
subsequent_mask Ran dddavid4real/HistGen/models/PlainTransformer_Modules.py
code served (permissive licence) · get_code("dd03765c7da9ca51")
transform_masks2regions Ran dddavid4real/HistGen/modules/histgen_module.py
code served (permissive licence) · get_code("ccfa98711a392912")
transform_sentences2tokens Ran dddavid4real/HistGen/modules/histgen_module.py
code served (permissive licence) · get_code("4707a8b847159f66")
transform_tokens2regions Ran dddavid4real/HistGen/modules/histgen_module.py
code served (permissive licence) · get_code("7f66a357e52cacea")

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

Histopathology serves as the gold standard in cancer diagnosis, with clinical reports being vital in interpreting and understanding this process, guiding cancer treatment and patient care. The automation of histopathology report generation with deep learning stands to significantly enhance clinical efficiency and lessen the labor-intensive, time-consuming burden on pathologists in report writing. In pursuit of this advancement, we introduce HistGen, a multiple instance learning-empowered framework for histopathology report generation together with the first benchmark dataset for evaluation. Inspired by diagnostic and report-writing workflows, HistGen features two delicately designed modules, aiming to boost report generation by aligning whole slide images (WSIs) and diagnostic reports from local and global granularity. To achieve this, a local-global hierarchical encoder is developed for efficient visual feature aggregation from a region-to-slide perspective. Meanwhile, a cross-modal context module is proposed to explicitly facilitate alignment and interaction between distinct modalities, effectively bridging the gap between the extensive visual sequences of WSIs and corresponding highly summarized reports. Experimental results on WSI report generation show the proposed model outperforms state-of-the-art (SOTA) models by a large margin. Moreover, the results of fine-tuning our model on cancer subtyping and survival analysis tasks further demonstrate superior performance compared to SOTA methods, showcasing strong transfer learning capability. Dataset, model weights, and source code are available in https://github.com/dddavid4real/HistGen.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2403.05396")
get_code_for_paper("2403.05396")
have("2403.05396")

Connect an agent — have() is free.