Nicholas Konz, Yixin Zhang, Kevin Kramer, Maciej A. Mazurowski
We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Image segmentation foundation models (SFMs) like Segment Anything Model (SAM) have achieved impressive zero-shot and interactive segmentation across diverse domains. However, they struggle to segment objects with certain structures, particularly those with dense, tree-like morphology and low textural contrast from their surroundings. These failure modes are crucial for understanding the limitations of SFMs in real-world applications. To systematically study this issue, we introduce interpretable metrics quantifying object tree-likeness and textural separability. On carefully controlled synthetic experiments and real-world datasets, we show that SFM performance (\eg, SAM, SAM 2, HQ-SAM) noticeably correlates with these factors. We attribute these failures to SFMs misinterpreting local structure as global texture, resulting in over-segmentation or difficulty distinguishing objects from similar backgrounds. Notably, targeted fine-tuning fails to resolve this issue, indicating a fundamental limitation. Our study provides the first quantitative framework for modeling the behavior of SFMs on challenging structures, offering interpretable insights into their segmentation capabilities.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2412.04243")
get_code_for_paper("2412.04243")
have("2412.04243")
The run record, dated, one paper per request, free:
curl https://syntology.ai/api/ran/2412.04243.json
A badge for a README (the split and the date, never a ratio):
[](https://syntology.ai/paper/2412.04243)
Connect an agent — have() is free.