SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2401.13246 · ACL · 2024

SEER: Facilitating Structured Reasoning and Explanation via Reinforcement Learning

Yu Qiao, Chao Yang, Guoxin Chen, Yiming Qian, Kexin Tang, Fuying Ye

arXiv · PDF · Open in the Atlas

Code that ran

13 of 16 samples (functions and classes) lifted from this paper's linked repositories executed on a synthesized input in Syntology's jail (0 of them constructed an object rather than computing a result; 0 honoured the contract we drafted, 2 violated it and 0 have no contract verdict; 11 failures of our instrument; 3 no recorded run), measured 2026-11-22; repository links from the Papers with Code archive attributed under CC BY-SA, no affiliation; bodies inline only under a permissive licence as GitHub detects it. Not a reproduction of the paper's results.

"Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results. This record, dated, as JSON: /api/ran/2401.13246.json · badge: /api/ran/2401.13246.svg

Each repository's role is a label: a classifier's output recorded on the link, or, where no role is recorded, what the Papers with Code archive marked. The line under it names its basis. Neither says who wrote the code.

RepositoryRoleRan
chen-gx/seer canonical (classifier label)
classifier basis: census
7 ran · 1 no recorded run
Chen-GX/SEER community
Papers with Code archive
5 ran · 2 no recorded run
copy not recorded community
not in the paper's declared repositories
1 ran · 0 no recorded run
FunctionStatusWhere it lives
add_fullstop Ran
ran; our contract draft was wrong, not the code
Chen-GX/SEER/etree_task1/src/RL_etree.py
code served (permissive licence) · get_code("37840dac8fbe9984")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned str. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch

def add_fullstop(sent):
    if sent.endswith('.'):
        return sent
    else:
        return sent + '.'

ARGS = {
    'sent': 'Hello world'
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type str — a str output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a str output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
check_nan_and_inf Ran
ran; our contract draft was wrong, not the code
chen-gx/seer/etree_task1/src/policy_net.py
code served (permissive licence) · get_code("f327a42b68b8350d")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned bool, ndim 0, dtype family float, shape () as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch

torch.manual_seed(0)

# Create a simple tensor to pass to check_nan_and_inf
# The function checks for NaN and Inf values in a tensor
tensor = torch.randn(2, 4)

ARGS = {
    "tensor": tensor
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type bool, ndim 0, dtype family float, shape () — shape () as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
decapitalize Ran
ran; our contract draft was wrong, not the code
Chen-GX/SEER/etree_task1/src/RL_etree.py
code served (permissive licence) · get_code("97e18d0b698dd4c8")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned str. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch

# The function decapitalize takes a single parameter: sent (a string)
# It converts the first character to lowercase and returns the modified string

ARGS = {
    "sent": "Hello World"
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type str — a str output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a str output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
get_task1 Ran
ran; our contract draft was wrong, not the code
chen-gx/seer/etree_task1/src/utils.py
code served (permissive licence) · get_code("2876f6baff3466a8")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned list, ndim 1, dtype family float; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import os
import os.path as osp
import json
import tempfile
import shutil

# Create a temporary directory for test data
temp_dir = tempfile.mkdtemp()
data_dir = osp.join(temp_dir, 'task_1')
os.makedirs(data_dir, exist_ok=True)

# Create a sample JSONL file for the data split
data_split = 'train'
data_file = osp.join(data_dir, f'{data_split}.jsonl')

# Create sample data that matches what the function expects
sample_items = [
    {
        'meta': {
            'triples': {
                'sent1': 'The cat is on the mat.',
                'sent2': 'The mat is in the house.'
            }
        },
        'hypothesis': 'The cat is in the house.',
        'proof': 'sent1->sent2: The cat is in the house.; sent2->hypothesis'
    },
    {
        'meta': {
            'triples': {
                'fact1': 'Dogs are animals.',
                'fact2': 'Animals need food.'
            }
        },
        'hypothesis': 'Dogs need food.',
        'proof': 'fact1->fact2: Dogs need food.; fact2->hypothesis'
    }
]

with open(data_file, 'w') as f:
    for item in sample_items:
        f.write(json.dumps(item) + '\n')

# Create the args object
class Args:
    def __init__(self):
        self.data_dir = temp_dir

args = Args()

# Create the train list with empty dicts to be populated
train = [{} for _ in range(len(sample_items))]

# Define ARGS dict
ARGS = {
    'args': args,
    'train': train,
    'data_split': data_split
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type list, ndim 1, dtype family float — a list output has no array shape of its own; none is stated
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
get_task2 Ran
ran; our contract draft was wrong, not the code
chen-gx/seer/etree_task1/src/utils.py
code served (permissive licence) · get_code("2809af8aa84d6463")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned list, ndim 1, dtype family float; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import os
import os.path as osp
import json
import tempfile
import shutil

# Create a temporary directory structure for the test
temp_dir = tempfile.mkdtemp()
task_2_dir = osp.join(temp_dir, 'task_2')
os.makedirs(task_2_dir, exist_ok=True)

# Create sample JSONL data files
train_data_path = osp.join(task_2_dir, 'train.jsonl')
with open(train_data_path, 'w') as f:
    # Write sample data for train split
    item1 = {
        'meta': {
            'triples': {
                'sent1': 'John is a student',
                'sent2': 'Students study books'
            }
        },
        'hypothesis': 'John studies books',
        'proof': 'sent1->sent2: John studies books; sent2->hypothesis: John studies books'
    }
    f.write(json.dumps(item1) + '\n')
    
    item2 = {
        'meta': {
            'triples': {
                'sent3': 'Mary is tall',
                'sent4': 'Tall people play basketball'
            }
        },
        'hypothesis': 'Mary plays basketball',
        'proof': 'sent3->sent4: Mary plays basketball'
    }
    f.write(json.dumps(item2) + '\n')

# Create sample JSONL data files for other splits
test_data_path = osp.join(task_2_dir, 'test.jsonl')
with open(test_data_path, 'w') as f:
    item3 = {
        'meta': {
            'triples': {
                'sent5': 'Cats are animals'
            }
        },
        'hypothesis': 'Cats exist',
        'proof': 'sent5->hypothesis: Cats exist'
    }
    f.write(json.dumps(item3) + '\n')

# Create args object
class Args:
    def __init__(self, data_dir):
        self.data_dir = data_dir

# Initialize train list with placeholder dicts
train = [{}, {}]

# Create ARGS dictionary
ARGS = {
    'args': Args(temp_dir),
    'train': train,
    'data_split': 'train'
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type list, ndim 1, dtype family float — a list output has no array shape of its own; none is stated
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
load_data Ran
ran; our contract draft was wrong, not the code
chen-gx/seer/etree_task1/src/utils.py
code served (permissive licence) · get_code("91ef4f10af2f0360")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned tuple, ndim 2, dtype family float; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import os
import json
import tempfile
import os.path as osp

# Create a temporary directory structure with mock data files
tmpdir = tempfile.mkdtemp()
task_dir = osp.join(tmpdir, 'task_1')
os.makedirs(task_dir, exist_ok=True)

# Create mock JSONL files for train, dev, test
def create_mock_jsonl(filepath, num_samples=2):
    with open(filepath, 'w') as f:
        for i in range(num_samples):
            item = {
                "meta": {
                    "triples": {
                        "sent0": "Alice is a person",
                        "sent1": "Bob is a student",
                        "int0": "Alice is a student"
                    }
                },
                "hypothesis": "Alice is a student",
                "proof": "sent0->int0: Alice is a person implies Alice is a student; int0->hypothesis: Alice is a student"
            }
            f.write(json.dumps(item) + '\n')

for split in ['train', 'dev', 'test']:
    create_mock_jsonl(osp.join(task_dir, f'{split}.jsonl'), num_samples=2)

# Create a mock args class with required attributes
class Args:
    def __init__(self, data_dir, task, debug_max_num):
        self.data_dir = data_dir
        self.task = task
        self.debug_max_num = debug_max_num

ARGS = {
    'args': Args(data_dir=tmpdir, task='task_1', debug_max_num=0),
    'agent': None,
    'return_dict': False
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type tuple, ndim 2, dtype family float — a tuple output has no array shape of its own; none is stated
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
normalize Ran
ran; our contract draft was wrong, not the code
Chen-GX/SEER/etree_task1/src/RL_etree.py
code served (permissive licence) · get_code("cf0a06f7868d1697")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned str. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

# Setup code for normalize function
import torch

# The normalize function takes a single parameter: sent (a string)
# It performs string operations: endswith, lower, strip, concatenation
# We need to provide a valid string argument

ARGS = {
    "sent": "Hello World"
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type str — a str output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a str output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
normalize_answer Ran
ran; our contract draft was wrong, not the code
this paper's copy was not recorded; identical code first harvested from borealisai/radar-decoding
pointer only; the stored copy get_code serves, from borealisai/radar-decoding, is pointer only (licence: LGPL-3.0) · get_code("e7e75981cb464788")
How it ran

The body of this sample is withheld: this paper's copy of the code was not recorded (its link names no repository or source path), so no licence was read for it. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

normalize_sentence Ran
ran; our contract draft was wrong, not the code
chen-gx/seer/etree_task1/src/reward.py
code served (permissive licence) · get_code("687eee6f12fa2fbe")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned str. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import numpy as np

# Analyze the function:
# normalize_sentence(sent) takes a single parameter 'sent'
# It calls string methods: replace(), lower(), strip()
# So 'sent' must be a string

# Create a test sentence with various patterns the function handles
test_sentence = "Hello  world.  This is ( a test )\n"

ARGS = {
    'sent': test_sentence
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type str — a str output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a str output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
parse_entailment_step_proof Ran
ran; our contract draft was wrong, not the code
chen-gx/seer/etree_task1/src/reward.py
code served (permissive licence) · get_code("07b94a2626ece603")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned tuple. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import numpy as np

def normalize_sentence(sent):
    return sent.replace('  ', ' ').replace('.', '').replace('\n', '').replace('( ', '').replace(' )', '').lower().strip()

def parse_entailment_step_proof(proof: str, gold_json_record: dict, slot_json_record=None):
    sentences = []
    inferences = []
    int_to_all_ancestors = dict()
    int_to_all_ancestors_list = []
    relevant_sentences = set()
    id_to_int = dict()
    if '[STEP]' in proof:
        proof_steps = proof.split('[STEP] ', maxsplit=-1)
    else:
        proof_steps = proof.split(';', maxsplit=-1)
    for p_step in proof_steps:
        p_parts = p_step.strip().split(':', 1)
        step = p_parts[0]
        int_str = ''
        if len(p_parts) == 2:
            int_str = p_parts[1].strip()
        if step:
            step_parts = step.split(' -> ', 1)
            if len(step_parts) == 2:
                rhs = step_parts[1].strip()
                if len(int_str) == 0:
                    int_str = slot_json_record['meta']['triples'][rhs]
                elif int_str != slot_json_record['meta']['triples'][rhs]:
                    int_str = slot_json_record['meta']['triples'][rhs]
                if '&' in step_parts[0]:
                    lhs_ids = step_parts[0].split('&')
                else:
                    lhs_ids = step_parts[0].split(',')
                all_ancestors = set()
                lhs_ids = [lid.strip() for lid in lhs_ids]
                for lid in lhs_ids:
                    if 'sent' in lid:
                        relevant_sentences.add(lid)
                        all_ancestors.add(lid)
                    else:
                        their_ancestors = int_to_all_ancestors.get(lid, set())
                        all_ancestors = all_ancestors.union(their_ancestors)
                sorted_lhs_ids = sorted(lhs_ids)
                sorted_lhs = ' & '.join(sorted_lhs_ids)
                sentences.append(f'{sorted_lhs} -> {rhs}')
                if rhs == 'hypothesis':
                    int_str = normalize_sentence(gold_json_record['hypothesis'])
                id_to_int[rhs] = int_str
                int_to_all_ancestors[rhs] = all_ancestors
                int_to_all_ancestors_list.append({'int': rhs, 'ancestors': list(all_ancestors)})
                inferences.append({'lhs': sorted_lhs_ids, 'rhs': rhs})
    return (sentences, inferences, int_to_all_ancestors_list, relevant_sentences, id_to_int)


proof = "sent1 & sent2 -> int1: test sentence; int1, sent3 -> hypothesis"

gold_json_record = {
    "hypothesis": "This is the final conclusion."
}

slot_json_record = {
    "meta": {
        "triples": {
            "int1": "intermediate step one",
            "hypothesis": "the hypothesis text"
        }
    }
}

ARGS = {
    "proof": proof,
    "gold_json_record": gold_json_record,
    "slot_json_record": slot_json_record
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type tuple — a tuple output has no array shape of its own; none is stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
same_sent Ran
ran, violated the contract we drafted
Chen-GX/SEER/etree_task1/src/RL_etree.py
code served (permissive licence) · get_code("252ca6659b772750")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned bool, ndim 0, dtype family float, shape () as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import re
import string

def normalize_answer(s):
    """Lower text and remove punctuation, articles and extra whitespace."""

    def remove_articles(text):
        return re.sub('\\b(a|an|the)\\b', ' ', text)

    def white_space_fix(text):
        return ' '.join(text.split())

    def remove_punc(text):
        exclude = set(string.punctuation)
        return ''.join((ch for ch in text if ch not in exclude))

    def lower(text):
        return text.lower()
    return white_space_fix(remove_articles(remove_punc(lower(s))))

def same_sent(sent1, sent2):
    return normalize_answer(sent1) == normalize_answer(sent2)

ARGS = {
    "sent1": "The quick brown fox jumps over the lazy dog.",
    "sent2": "the quick brown fox jumps over the lazy dog"
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type bool, ndim 0, dtype family float, shape () — shape () as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
sort_key Ran
ran; our contract draft was wrong, not the code
Chen-GX/SEER/etree_task1/src/RL_etree.py
code served (permissive licence) · get_code("5218be161f3c9262")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned tuple, ndim 1, dtype family float; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import re

def sort_key(item):
    match = re.match('(\\D+)(\\d+)', item[0])
    prefix = match.group(1)
    number = int(match.group(2))
    return (prefix, number)

# The function takes a single parameter 'item' which is indexed with [0]
# It expects item[0] to be a string that matches the pattern (letters)(digits)
# The function returns a tuple of (prefix_string, number_int)

ARGS = {
    'item': ('abc123',)
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type tuple, ndim 1, dtype family float — a tuple output has no array shape of its own; none is stated
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
str2bool Ran
ran, violated the contract we drafted
chen-gx/seer/etree_task1/src/arguments.py
pointer only (licence: MIT); the stored copy get_code serves, from Chen-GX/ToolEVO, is pointer only (licence: NONE) · get_code("92a5f0d0912c3a91")
How it ran

The body of this sample is withheld: this paper's link allows the body inline (MIT), but get_code serves the stored copy, first harvested from Chen-GX/ToolEVO, and GitHub detected no licence in that repository when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

State Not yet run
unverified (harvested, no recorded run)
Chen-GX/SEER/etree_task1/src/RL_etree.py
code served (permissive licence) · get_code("7dab5599647f8485")
How it ran

no invocation recorded.

eTree Not yet run
unverified (harvested, no recorded run)
Chen-GX/SEER/etree_task1/src/RL_etree.py
code served (permissive licence) · get_code("01ce62aa5e1a87d2")
How it ran

no invocation recorded.

score_aligned_entail_tree_proof Not yet run
unverified (harvested, no recorded run)
chen-gx/seer/etree_task1/src/reward.py
code served (permissive licence) · get_code("27dff68f0e4774cb")
How it ran

no invocation recorded.

Also present in other papers' code

Of 16 samples lifted from this paper's linked repositories, 16 carry a function identity and 0 do not. Of the 16 distinct functions with an identity, 2 are also present in other papers whose code we hold (2 with at least one paper dated earlier, 1 the same month, 2 later) and 14 are only here among the 52,977 papers whose code Syntology holds (as of 2026-09-26).

Only here means: among the 52,977 papers whose code Syntology holds (51,887 with lifted samples, 1,090 through their assemblies only), as of 2026-09-26, no other paper's lifted samples or current assembly bricks contain an identical function body after normalisation.

FunctionAlso present inEarliest-datedOthers named
normalize_answer
brick of this paper's assembly: helper
133 other papers
57 dated earlier, 1 the same month, 74 later, 1 undated · 1 only as an assembly brick
1508.06615 (2015-08)
Character-Aware Neural Language Models
1606.05250 (2016-06)
SQuAD: 100,000+ Questions for Machine Comprehension of Text
1608.07905 (2016-08)
Machine Comprehension Using Match-LSTM and Answer Pointer
and 130 more not named here
str2bool 2 other papers
1 dated earlier, 1 later
2211.06679 (2022-11)
AltCLIP: Altering the Language Encoder in CLIP for Extended Language Capabilities
2410.06617 (2024-10)
Learning Evolving Tools for Large Language Models

Presence, not direction: an identical function body in two papers' repositories does not say who wrote it, which came from which, or that either paper's method rests on it. Matching is exact after normalisation, so a renamed or edited function is not matched. Dates are when papers appeared, not when code was written. Only .py files the harvester lifted are covered; notebooks are not read. A new-style arXiv id YYMM.NNNN dates a paper to the month 20YY-MM; otherwise Paper.year dates it to the year; otherwise it is undated. Dates are when the paper appeared, not when the code was written. Read 2026-11-22.

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

Elucidating the reasoning process with structured explanations from question to answer is crucial, as it significantly enhances the interpretability, traceability, and trustworthiness of question-answering (QA) systems. However, structured explanations demand models to perform intricately structured reasoning, which poses great challenges. Most existing methods focus on single-step reasoning through supervised learning, ignoring logical dependencies between steps. Moreover, existing reinforcement learning (RL) based methods overlook the structured relationships, underutilizing the potential of RL in structured reasoning. In this paper, we propose SEER, a novel method that maximizes a structure-based return to facilitate structured reasoning and explanation. Our proposed structure-based return precisely describes the hierarchical and branching structure inherent in structured reasoning, effectively capturing the intricate relationships between different reasoning steps. In addition, we introduce a fine-grained reward function to meticulously delineate diverse reasoning steps. Extensive experiments show that SEER significantly outperforms state-of-theart methods, achieving an absolute improvement of 6.9% over RL-based methods on En-tailmentBank, a 4.4% average improvement on STREET benchmark, and exhibiting outstanding efficiency and cross-dataset generalization performance. Our code is available at https://github.com/Chen-GX/SEER.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2401.13246")
get_code_for_paper("2401.13246")
have("2401.13246")

The run record, dated, one paper per request, free:

curl https://syntology.ai/api/ran/2401.13246.json

A badge for a README (the split and the date, never a ratio):

[![Syntology run record](https://syntology.ai/api/ran/2401.13246.svg)](https://syntology.ai/paper/2401.13246)

Connect an agent — have() is free.