SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2406.06512 · 2024

Merlin: A Computed Tomography Vision-Language Foundation Model and Dataset

arXiv · PDF · Open in the Atlas

Code that ran

8 of 9 samples (functions and classes) lifted from this paper's linked repositories executed on a synthesized input in Syntology's jail (0 of them constructed an object rather than computing a result; 0 honoured the contract we drafted, 0 violated it and 7 have no contract verdict; 1 failure of our instrument; 1 no recorded run), measured 2026-11-22; repository links from the Papers with Code archive attributed under CC BY-SA, no affiliation; bodies inline only under a permissive licence as GitHub detects it. Not a reproduction of the paper's results.

"Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results. This record, dated, as JSON: /api/ran/2406.06512.json · badge: /api/ran/2406.06512.svg

Each repository's role is a label: a classifier's output recorded on the link, or, where no role is recorded, what the Papers with Code archive marked. The line under it names its basis. Neither says who wrote the code.

RepositoryRoleRan
stanfordmimi/merlin community (classifier label)
classifier basis not recorded on the link
8 ran · 1 no recorded run
FunctionStatusWhere it lives
build_datalist Ran
ran on a synthesized input
stanfordmimi/merlin/documentation/Merlin_CVPR26CTFM/extract_feat_LP.py
code served (permissive licence) · get_code("bbc014ffe2cb5cdb")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned list, ndim 1, dtype family float; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import os
import tempfile
import shutil

# Create temporary directories with mock .nii.gz files
temp_imgs_dir = tempfile.mkdtemp()
temp_masks_dir = tempfile.mkdtemp()

# Create some mock .nii.gz files in images directory
for i in range(3):
    fname = f"case_{i:02d}.nii.gz"
    open(os.path.join(temp_imgs_dir, fname), 'w').close()

# Create corresponding mask files
for i in range(3):
    fname = f"case_{i:02d}.nii.gz"
    open(os.path.join(temp_masks_dir, fname), 'w').close()

# Also add a file in imgs that doesn't have a mask to test filtering
open(os.path.join(temp_imgs_dir, "case_99.nii.gz"), 'w').close()

ARGS = {
    'imgs_path': temp_imgs_dir,
    'masks_path': temp_masks_dir,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type list, ndim 1, dtype family float — a list output has no array shape of its own; none is stated
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
escape_cell Ran
ran on a synthesized input
stanfordmimi/merlin/community/validate.py
code served (permissive licence) · get_code("2b41135de70dd39a")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned str. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch

def escape_cell(text: str) -> str:
    """Make a string safe to drop inside a Markdown table cell."""
    return ' '.join(str(text).split()).replace('|', '\\|')

ARGS = {
    'text': 'hello | world  with   extra spaces'
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type str — a str output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a str output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
inflate_batch_norm Ran
ran; our contract draft was wrong, not the code
stanfordmimi/merlin/merlin/models/inflate.py
code served (permissive licence) · get_code("457cc8c6dc365fbc")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned BatchNorm2d. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn

torch.manual_seed(0)

# Create a BatchNorm2d instance
batch2d = nn.BatchNorm2d(num_features=4)

# The function expects a batch2d parameter (a BatchNorm2d module)
ARGS = {
    'batch2d': batch2d
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type BatchNorm2d — a BatchNorm2d output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a BatchNorm2d output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
inflate_conv Ran
ran on a synthesized input
stanfordmimi/merlin/merlin/models/inflate.py
code served (permissive licence) · get_code("aeb746dad82eaf48")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned Conv3d. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn
from torch.nn import Parameter

torch.manual_seed(0)

# Create a simple Conv2d layer to pass as the conv2d argument
# Using in_channels=3, out_channels=4, kernel_size=3 for a typical case
conv2d_layer = nn.Conv2d(in_channels=3, out_channels=4, kernel_size=3, padding=1, stride=1, dilation=1)

ARGS = {
    "conv2d": conv2d_layer,
    "time_dim": 3,
    "time_padding": 0,
    "time_stride": 1,
    "time_dilation": 1,
    "center": False,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Conv3d — a Conv3d output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a Conv3d output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
inflate_linear Ran
ran on a synthesized input
stanfordmimi/merlin/merlin/models/inflate.py
code served (permissive licence) · get_code("57ec9531440b9006")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned Linear. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
from torch import nn
from torch.nn import Parameter

torch.manual_seed(0)

# Create a small 2D linear layer to inflate
# in_features=4, out_features=3
linear2d = nn.Linear(in_features=4, out_features=3)

# Define time_dim
time_dim = 2

ARGS = {
    "linear2d": linear2d,
    "time_dim": time_dim,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Linear — a Linear output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a Linear output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
render_links Ran
ran on a synthesized input
stanfordmimi/merlin/community/validate.py
code served (permissive licence) · get_code("a7cd3d0cba5c0c42")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned str. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

LINK_KEYS = ('homepage', 'code', 'data', 'paper', 'demo', 'docs')

LINK_LABELS = {'homepage': 'Project', 'code': 'Code', 'data': 'Data', 'paper': 'Paper', 'demo': 'Demo', 'docs': 'Docs'}

def render_links(links: dict) -> str:
    """One label per distinct URL.

    Projects commonly point 'homepage' and 'code' at the same repository; show
    that URL once, labelled with the more specific of the two.
    """
    chosen: dict[str, str] = {}
    for key in LINK_KEYS:
        url = str(links.get(key, '')).strip()
        if not url:
            continue
        if url not in chosen:
            chosen[url] = key
        elif chosen[url] == 'homepage':
            chosen[url] = key
    ordered = sorted(chosen.items(), key=lambda item: LINK_KEYS.index(item[1]))
    return ' · '.join((f'[{LINK_LABELS[key]}]({url})' for url, key in ordered))


# Test case 1: Multiple links with some duplicates
links_example = {
    'homepage': 'https://github.com/example/project',
    'code': 'https://github.com/example/project',
    'paper': 'https://arxiv.org/abs/2101.00001',
    'demo': 'https://example.com/demo',
}

ARGS = {
    'links': links_example
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type str — a str output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a str output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
sanitize_report Ran
ran on a synthesized input
stanfordmimi/merlin/merlin/models/build.py
code served (permissive licence) · get_code("9d2d06ffb8b421c7")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned str. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import numpy as np

ARGS = {
    "report": "The Quick Brown Fox Jumps Over the Lazy Dog! This is a test report."
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
nltk, numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type str — a str output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a str output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
validate_entry Ran
ran on a synthesized input
stanfordmimi/merlin/community/validate.py
code served (permissive licence) · get_code("8af2c98983431d38")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned list, ndim 1, dtype family float; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import datetime
from pathlib import Path

# Create a valid entry dictionary
valid_entry = {
    'name': 'Test Project',
    'description': 'A test project description that is reasonably short.',
    'category': 'model',
    'authors': [
        {
            'name': 'John Doe',
            'affiliation': 'Test University',
            'email': 'john@example.com'
        }
    ],
    'license': 'MIT',
    'links': {
        'homepage': 'https://example.com',
        'code': 'https://github.com/example/repo',
        'paper': 'https://arxiv.org/abs/1234.5678'
    },
    'added': '2026-09-11',
    'tags': ['machine-learning', 'pytorch'],
    'merlin_components': ['component1', 'component2'],
    'citation': '@article{example,\n  title={Example},\n  year={2026}\n}',
    'contact': 'contact@example.com'
}

# Create a valid Path object pointing to a yaml file
valid_path = Path('my-project.yaml')

ARGS = {
    'path': valid_path,
    'entry': valid_entry
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type list, ndim 1, dtype family float — a list output has no array shape of its own; none is stated
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
download_file Not yet run
unverified (harvested, no recorded run)
stanfordmimi/merlin/merlin/utils/huggingface_download.py
code served (permissive licence) · get_code("d9b7e589764478c3")
How it ran

no invocation recorded.

Also present in other papers' code

Of 9 samples lifted from this paper's linked repositories, 9 carry a function identity and 0 do not. Of the 9 distinct functions with an identity, 1 are also present in other papers whose code we hold (1 with at least one paper dated earlier, 0 the same month, 0 later) and 8 are only here among the 52,977 papers whose code Syntology holds (as of 2026-09-26).

Only here means: among the 52,977 papers whose code Syntology holds (51,887 with lifted samples, 1,090 through their assemblies only), as of 2026-09-26, no other paper's lifted samples or current assembly bricks contain an identical function body after normalisation.

FunctionAlso present inEarliest-datedOthers named
inflate_batch_norm 1 other paper
1 dated earlier
2101.10241 (2021-01)
RGB-D Salient Object Detection via 3D Convolutional Neural Networks

Presence, not direction: an identical function body in two papers' repositories does not say who wrote it, which came from which, or that either paper's method rests on it. Matching is exact after normalisation, so a renamed or edited function is not matched. Dates are when papers appeared, not when code was written. Only .py files the harvester lifted are covered; notebooks are not read. A new-style arXiv id YYMM.NNNN dates a paper to the month 20YY-MM; otherwise Paper.year dates it to the year; otherwise it is undated. Dates are when the paper appeared, not when the code was written. Read 2026-11-22.

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

The large volume of abdominal computed tomography (CT) scans coupled with the shortage of radiologists have intensified the need for automated medical image analysis tools. Previous state-of-the-art approaches for automated analysis leverage vision-language models (VLMs) that jointly model images and radiology reports. However, current medical VLMs are generally limited to 2D images and short reports. Here to overcome these shortcomings for abdominal CT interpretation, we introduce Merlin, a 3D VLM that learns from volumetric CT scans, electronic health record data and radiology reports. This approach is enabled by a multistage pretraining framework that does not require additional manual annotations. We trained Merlin using a high-quality clinical dataset of paired CT scans (>6 million images from 15,331 CT scans), diagnosis codes (>1.8 million codes) and radiology reports (>6 million tokens). We comprehensively evaluated Merlin on 6 task types and 752 individual tasks that covered diagnostic, prognostic and quality-related tasks. The non-adapted (off-the-shelf) tasks included zero-shot classification of findings (30 findings), phenotype classification (692 phenotypes) and zero-shot cross-modal retrieval (image-to-findings and image-to-impression). The model-adapted tasks included 5-year chronic disease prediction (6 diseases), radiology report generation and 3D semantic segmentation (20 organs). We validated Merlin at scale, with internal testing on 5,137 CT scans and external testing on 44,098 CT scans from 3 independent sites and 2 public datasets. The results demonstrated high generalization across institutions and anatomies. Merlin outperformed 2D VLMs, CT foundation models and off-the-shelf radiology models. We also release our trained models, code, and dataset, available at: https://github.com/StanfordMIMI/Merlin.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2406.06512")
get_code_for_paper("2406.06512")
have("2406.06512")

The run record, dated, one paper per request, free:

curl https://syntology.ai/api/ran/2406.06512.json

A badge for a README (the split and the date, never a ratio):

[![Syntology run record](https://syntology.ai/api/ran/2406.06512.svg)](https://syntology.ai/paper/2406.06512)

Connect an agent — have() is free.