We lifted 3 functions out of this paper's own repositories and ran 3 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| gkw0010/endochat | canonical | 3 of 3 |
| Function | Status | Where it lives |
|---|---|---|
| calc_metric_single_phrase | Ran | gkw0010/endochat/accessory/get_evaluation_result.py pointer only (licence: NONE) · get_code("15f1f93594928329") |
| metrics_acc_fscore | Ran | gkw0010/endochat/accessory/get_evaluation_result.py pointer only (licence: NONE) · get_code("1826f67bf3ac59eb") |
| parse_bbox_string | Ran | gkw0010/endochat/accessory/get_evaluation_result.py pointer only (licence: NONE) · get_code("4cbf335103f9ed8b") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Recently, Multimodal Large Language Models (MLLMs) have demonstrated their immense potential in computer-aided diagnosis and decision-making. In the context of robotic-assisted surgery, MLLMs can serve as effective tools for surgical training and guidance. However, there is still a lack of MLLMs specialized for surgical scene understanding in clinical applications. In this work, we introduce EndoChat to address various dialogue paradigms and subtasks in surgical scene understanding that surgeons encounter. To train our EndoChat, we construct the Surg-396K dataset through a novel pipeline that systematically extracts surgical information and generates structured annotations based on collected large-scale endoscopic surgery datasets. Furthermore, we introduce a multi-scale visual token interaction mechanism and a visual contrast-based reasoning mechanism to enhance the model's representation learning and reasoning capabilities. Our model achieves state-of-the-art performance across five dialogue paradigms and eight surgical scene understanding tasks. Additionally, we conduct evaluations with professional surgeons, most of whom provide positive feedback on collaborating with EndoChat. Overall, these results demonstrate that our EndoChat has great potential to significantly advance training and automation in robotic-assisted surgery.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2501.11347")
get_code_for_paper("2501.11347")
have("2501.11347")
Connect an agent — have() is free.