SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2602.07892 · 2026

Safety Alignment as Continual Learning: Mitigating the Alignment Tax via Orthogonal Gradient Projection

Hang Su, Jun Zhu, Siyuan Zhang, Yi Zhong, Liyuan Wang, Guanglong Sun, Training Pipeline

arXiv · PDF · Open in the Atlas

Code that ran

16 of 18 samples (functions and classes) lifted from this paper's linked repositories executed on a synthesized input in Syntology's jail (0 of them constructed an object rather than computing a result; 1 honoured the contract we drafted, 0 violated it and 0 have no contract verdict; 15 failures of our instrument; 2 no recorded run), measured 2026-11-22; bodies inline only under a permissive licence as GitHub detects it. Not a reproduction of the paper's results.

"Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results. This record, dated, as JSON: /api/ran/2602.07892.json · badge: /api/ran/2602.07892.svg

Each repository's role is a label: a classifier's output recorded on the link, or, where no role is recorded, what the Papers with Code archive marked. The line under it names its basis. Neither says who wrote the code.

RepositoryRoleRan
SunGL001/OGPSA canonical (classifier label)
classifier basis: census
16 ran · 2 no recorded run
FunctionStatusWhere it lives
build_gpt_eval_messages Ran
ran; our contract draft was wrong, not the code
SunGL001/OGPSA/eval/HalluQA.py
code served (permissive licence) · get_code("4af073ea30776bc7")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned list, ndim 1, dtype family float; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import numpy as np

torch.manual_seed(0)

# sample dict - contains model_response key
sample = {
    'model_response': '根据提供的信息,这个问题的答案是相关的内容。',
    'other_field': 'some_value'
}

# ref dict - contains Question and Best Answer keys
ref = {
    'Question': '什么是人工智能?',
    'Best Answer': '人工智能是计算机科学的一个分支,致力于创建能够执行通常需要人类智能的任务的系统。',
    'Best Answer1': '人工智能(AI)是指由人制造出来的机器所表现的智能。',
    'Best Answer2': 'AI是模拟人类智能过程的计算机科学技术。',
}

ARGS = {
    'sample': sample,
    'ref': ref
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type list, ndim 1, dtype family float — a list output has no array shape of its own; none is stated
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
build_gpt_eval_prompt Ran
ran; our contract draft was wrong, not the code
SunGL001/OGPSA/eval/AIME2024.py
code served (permissive licence) · get_code("89e5642dfceebefd")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned str. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import numpy as np

# Simple string-based arguments for the build_gpt_eval_prompt function
problem = "What is the sum of 5 and 3?"
response = "The sum of 5 and 3 is calculated as follows: 5 + 3 = 8. Therefore, the final answer is 8."
correct_answer = 8

ARGS = {
    "problem": problem,
    "response": response,
    "correct_answer": correct_answer
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type str — a str output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a str output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
build_gpt_eval_prompt Ran
ran; our contract draft was wrong, not the code
SunGL001/OGPSA/eval/AdvGLUE.py
code served (permissive licence) · get_code("0d72969c11a70528")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned str. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import numpy as np

# Define ARGS dict with valid arguments for build_gpt_eval_prompt
ARGS = {
    "question": "What is the sentiment of this text?",
    "response": "The sentiment is positive because the text contains many positive adjectives and uplifting language. The label is: 1"
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type str — a str output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a str output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
build_gpt_eval_prompt Ran
ran; our contract draft was wrong, not the code
SunGL001/OGPSA/eval/GPQA.py
code served (permissive licence) · get_code("1c27eb34b1f772b2")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned str. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import numpy as np

torch.manual_seed(0)

ARGS = {
    "question": "What is the capital of France?",
    "response": "The capital of France is Paris, so the answer is B.",
    "choices": {
        "A": "London",
        "B": "Paris",
        "C": "Berlin",
        "D": "Madrid"
    }
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type str — a str output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a str output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
build_gpt_eval_prompt Ran
ran; our contract draft was wrong, not the code
SunGL001/OGPSA/eval/HHH.py
code served (permissive licence) · get_code("24d523840feeeb28")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned str. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import numpy as np

torch.manual_seed(0)

# Define test arguments for build_gpt_eval_prompt
question = "What is the capital of France?"
response = "The capital of France is Paris, which is option A."

ARGS = {
    "question": question,
    "response": response,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type str — a str output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a str output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
detect_model_type_from_path Ran
ran; our contract draft was wrong, not the code
SunGL001/OGPSA/eval/AlpacaEval.py
code served (permissive licence) · get_code("7bc66882c50c5fef")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned str. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn

# The function takes a single string parameter that represents an input file path
# It detects the model type based on whether 'qwen' or 'llama' appears in the path

# Test case 1: Path containing 'qwen'
input_path_qwen = "/models/qwen-7b/model.bin"

# Test case 2: Path containing 'llama'
input_path_llama = "/models/llama-2-7b/weights.pt"

# Test case 3: Path with no model type indicator
input_path_unknown = "/models/custom_model/checkpoint.pth"

# We'll use the qwen path as the main test case
ARGS = {
    "input_path": "/models/qwen-7b/model.bin"
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type str — a str output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a str output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
extract_answer Ran
ran; our contract draft was wrong, not the code
SunGL001/OGPSA/eval/MMLU.py
code served (permissive licence) · get_code("b9ce5a37c260fee5")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned str. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import re

def extract_answer(response):
    """
    从模型响应中提取答案(A/B/C/D)
    优先查找 "Answer: X" 格式,如果没有则查找其他可能的格式
    """
    if not response:
        return None
    patterns = ['Answer:\\s*([A-D])', 'Answer:\\s*([A-D])\\)', 'answer:\\s*([A-D])', 'ANSWER:\\s*([A-D])']
    for pattern in patterns:
        match = re.search(pattern, response, re.IGNORECASE)
        if match:
            answer = match.group(1).upper()
            if answer in ['A', 'B', 'C', 'D']:
                return answer
    lines = response.strip().split('\n')
    for line in reversed(lines[-5:]):
        line = line.strip()
        if len(line) == 1 and line.upper() in ['A', 'B', 'C', 'D']:
            return line.upper()
        match = re.search('\\b([A-D])\\b', line, re.IGNORECASE)
        if match:
            answer = match.group(1).upper()
            if answer in ['A', 'B', 'C', 'D']:
                return answer
    match = re.search('\\b([A-D])\\b', response, re.IGNORECASE)
    if match:
        answer = match.group(1).upper()
        if answer in ['A', 'B', 'C', 'D']:
            return answer
    return None


ARGS = {
    "response": "The correct Answer: B is the best choice."
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type str — a str output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a str output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
extract_answer_from_response Ran
ran; our contract draft was wrong, not the code
SunGL001/OGPSA/eval/AIME2024.py
code served (permissive licence) · get_code("e704f14bfb984d51")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned str. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import re

def extract_answer_from_response(response):
    """从模型回答中提取答案(尝试多种模式)"""
    if not response:
        return None
    boxed_pattern = '\\\\boxed\\{(\\d+)\\}'
    match = re.search(boxed_pattern, response)
    if match:
        return match.group(1)
    boxed_pattern2 = 'boxed\\{(\\d+)\\}'
    match = re.search(boxed_pattern2, response)
    if match:
        return match.group(1)
    final_answer_patterns = ['(?:The\\s+answer\\s+is|Final\\s+answer|Answer|答案是)[:\\s]*(\\d+)', '(?:Therefore|Thus|Hence)[,\\s]+(?:the\\s+answer\\s+is|we\\s+get)[:\\s]*(\\d+)']
    for pattern in final_answer_patterns:
        match = re.search(pattern, response, re.IGNORECASE)
        if match:
            return match.group(1)
    lines = response.strip().split('\n')
    for line in reversed(lines[-5:]):
        numbers = re.findall('\\b(\\d+)\\b', line)
        if numbers:
            return numbers[-1]
    return None


# Test cases with different response formats
test_response_1 = "The answer is 42"
test_response_2 = "Therefore, we get: 123"
test_response_3 = "Some calculation...\nFinal result: 789"
test_response_4 = ""
test_response_5 = "boxed{55}"

ARGS = {
    "response": test_response_1
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type str — a str output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a str output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
extract_answer_from_response Ran
ran; our contract draft was wrong, not the code
SunGL001/OGPSA/eval/GPQA.py
code served (permissive licence) · get_code("996c1c7807871550")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned str. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import re

ANSWER_PATTERN_MULTICHOICE = '(?i)Answer\\s*:\\s*([A-D])'

def extract_answer_from_response(response):
    """从模型回答中提取多选题答案(A、B、C 或 D)"""
    if not response:
        return None
    match = re.search(ANSWER_PATTERN_MULTICHOICE, response)
    if match:
        answer = match.group(1).upper()
        if answer in ['A', 'B', 'C', 'D']:
            return answer
    patterns = ['(?:答案是|答案|Answer|answer|The answer is|The correct answer is)[:\\s]*([A-D])', '([A-D])(?:\\s*\\)|\\s*\\.|\\s*$|\\s*[。,,])', '选择\\s*([A-D])', '选项\\s*([A-D])']
    for pattern in patterns:
        match = re.search(pattern, response, re.IGNORECASE)
        if match:
            answer = match.group(1).upper()
            if answer in ['A', 'B', 'C', 'D']:
                return answer
    lines = response.strip().split('\n')
    for line in reversed(lines[-5:]):
        match = re.search('\\b([A-D])\\b', line, re.IGNORECASE)
        if match:
            answer = match.group(1).upper()
            if answer in ['A', 'B', 'C', 'D']:
                return answer
    return None


ARGS = {
    "response": "Answer: B"
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type str — a str output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a str output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
extract_label_from_response Ran
ran; our contract draft was wrong, not the code
SunGL001/OGPSA/eval/HHH.py
code served (permissive licence) · get_code("522a73213ca49776")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned str. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import re

def extract_label_from_response(response):
    """从响应中提取标签(A 或 B)"""
    if not response:
        return None
    patterns = ['\\b[Aa]\\b', '\\b[Bb]\\b', 'Choice:\\s*([ABab])', 'Option\\s+([ABab])']
    for pattern in patterns:
        match = re.search(pattern, response)
        if match:
            label = match.group(1) if len(match.groups()) > 0 else match.group(0)
            return label.upper()
    return None


ARGS = {
    "response": "The answer is A"
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type str — a str output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a str output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
extract_model_identifier Ran
ran; our contract draft was wrong, not the code
SunGL001/OGPSA/eval/AlpacaEval_pre.py
code served (permissive licence) · get_code("92c41ca5148de65a")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned str. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import re

def extract_model_identifier(response):
    """从 GPT 响应中提取模型标识符(m 或 M)"""
    if not response:
        return None
    response = response.strip()
    if response == 'm':
        return 'm'
    if response == 'M':
        return 'M'
    match_m = re.search('\\b[m]\\b', response, re.IGNORECASE)
    if match_m:
        matched_char = match_m.group(0)
        if matched_char == 'm':
            return 'm'
        elif matched_char == 'M':
            return 'M'
    if 'm' in response.lower():
        for i, char in enumerate(response):
            if char.lower() == 'm':
                return char
    return None

# Test cases for the function
test_response_1 = "m"
test_response_2 = "M"
test_response_3 = "The model is m"
test_response_4 = "Choose between M or N"
test_response_5 = ""
test_response_6 = None
test_response_7 = "  m  "

ARGS = {
    "response": "The answer is m or M"
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type str — a str output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a str output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
get_prompt Ran
ran; our contract draft was wrong, not the code
SunGL001/OGPSA/eval/AlpacaEval_pre.py
code served (permissive licence) · get_code("fa528f9eb5d2a2af")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned str. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch

torch.manual_seed(0)

# Analyzing the function signature and body:
# - instruction: a string used in the prompt construction
# - ans1: a string representing the model response (gets inserted into prompt)
# - ans2: a string representing the baseline output (gets inserted into prompt)
# All three parameters are strings that are concatenated into a larger prompt message.

ARGS = {
    'instruction': 'What is the capital of France?',
    'ans1': '"Paris is the capital of France."',
    'ans2': '"The capital city of France is Paris."'
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type str — a str output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a str output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
get_token_count Ran
ran, honoured the contract we drafted
SunGL001/OGPSA/eval/AlpacaEval.py
code served (permissive licence) · get_code("3a11d80fea8ca185")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned int, ndim 0, dtype family int, shape () as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import numpy as np

torch.manual_seed(0)

# Since get_token_count calls _tokenizer.encode(text, add_special_tokens=False)
# we need a minimal tokenizer stand-in that has an encode method
class MockTokenizer:
    def encode(self, text, add_special_tokens=False):
        # Simple mock: return a list of token ids based on text length
        # Each "word" becomes a token
        return list(range(len(text.split()) if text else 0))

# Create a mock tokenizer instance
mock_tokenizer = MockTokenizer()

# Build the arguments dictionary for get_token_count
ARGS = {
    "text": "hello world this is a test string",
    "model_type": "qwen"
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
numpy, torch, transformers — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type int, ndim 0, dtype family int, shape () — shape () as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
process_one Ran
ran; our contract draft was wrong, not the code
SunGL001/OGPSA/eval/AdvGLUE.py
code served (permissive licence) · get_code("b543cea785162e44")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned tuple, ndim 1, dtype family float; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import numpy as np

torch.manual_seed(0)

# Construct valid arguments for process_one(idx, d)
# The function expects:
# - idx: an integer index
# - d: a dictionary with keys 'prompt_to_model', 'prompt', 'model_response', 'task', 'label'

idx = 0

d = {
    'prompt_to_model': 'What is the answer?',
    'prompt': 'Fallback prompt',
    'model_response': 'The answer is 1',
    'task': 'classification',
    'label': 1
}

ARGS = {
    'idx': idx,
    'd': d
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type tuple, ndim 1, dtype family float — a tuple output has no array shape of its own; none is stated
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
process_one Ran
ran; our contract draft was wrong, not the code
SunGL001/OGPSA/eval/GPQA_Diamond.py
code served (permissive licence) · get_code("f4dc40dbd0a904af")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned tuple, ndim 1, dtype family float; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import re

ANSWER_PATTERN_MULTICHOICE = '(?i)Answer\\s*:\\s*([A-D])'

def extract_answer(response):
    """从响应中提取答案字母(参考 gpqa_eval.py 的 get_score 函数)"""
    if not response:
        return None
    match = re.search(ANSWER_PATTERN_MULTICHOICE, response)
    if match:
        return match.group(1).upper()
    return None

def process_one(idx, d):
    """单条样本的处理逻辑(线程函数)"""
    response = d.get('model_response', '').strip()
    correct_answer = d.get('correct_answer_letter', '').strip().upper()
    extracted_answer = extract_answer(response)
    correct = False
    if extracted_answer and correct_answer:
        correct = extracted_answer == correct_answer
    d['extracted_answer'] = extracted_answer
    d['correct_answer'] = correct_answer
    d['correct'] = correct
    d['score'] = 1 if correct else 0
    return (idx, d)


# Setup code to construct valid arguments for process_one
idx = 0
d = {
    'model_response': 'The answer is: Answer: B',
    'correct_answer_letter': 'B',
}

ARGS = {
    'idx': idx,
    'd': d,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type tuple, ndim 1, dtype family float — a tuple output has no array shape of its own; none is stated
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
process_one Ran
ran; our contract draft was wrong, not the code
SunGL001/OGPSA/eval/MMLU.py
code served (permissive licence) · get_code("0364cbd97c9ad0de")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned tuple, ndim 1, dtype family float; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import re

def extract_answer(response):
    """
    从模型响应中提取答案(A/B/C/D)
    优先查找 "Answer: X" 格式,如果没有则查找其他可能的格式
    """
    if not response:
        return None
    patterns = ['Answer:\\s*([A-D])', 'Answer:\\s*([A-D])\\)', 'answer:\\s*([A-D])', 'ANSWER:\\s*([A-D])']
    for pattern in patterns:
        match = re.search(pattern, response, re.IGNORECASE)
        if match:
            answer = match.group(1).upper()
            if answer in ['A', 'B', 'C', 'D']:
                return answer
    lines = response.strip().split('\n')
    for line in reversed(lines[-5:]):
        line = line.strip()
        if len(line) == 1 and line.upper() in ['A', 'B', 'C', 'D']:
            return line.upper()
        match = re.search('\\b([A-D])\\b', line, re.IGNORECASE)
        if match:
            answer = match.group(1).upper()
            if answer in ['A', 'B', 'C', 'D']:
                return answer
    match = re.search('\\b([A-D])\\b', response, re.IGNORECASE)
    if match:
        answer = match.group(1).upper()
        if answer in ['A', 'B', 'C', 'D']:
            return answer
    return None

def process_one(idx, d):
    """单条样本的处理逻辑(线程函数)"""
    response = d.get('model_response', '').strip()
    label = d.get('label') or d.get('correct_answer_letter')
    extracted_label = extract_answer(response)
    if extracted_label is None:
        extracted_label = 'NONE'
    if label:
        label = str(label).upper().strip()
        correct = extracted_label != 'NONE' and extracted_label == label
    else:
        correct = False
    d['predicted_label'] = extracted_label
    d['correct'] = correct
    d['score'] = 1 if correct else 0
    return (idx, d)


# Setup code
sample_dict = {
    'model_response': 'The answer is Answer: B because of the following reasoning.',
    'label': 'B',
    'correct_answer_letter': None
}

idx_value = 0

ARGS = {
    'idx': idx_value,
    'd': sample_dict
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type tuple, ndim 1, dtype family float — a tuple output has no array shape of its own; none is stated
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
extract_answer Not yet run
unverified (harvested, no recorded run)
SunGL001/OGPSA/eval/GPQA_Diamond.py
code served (permissive licence) · get_code("a7b81f2f5969d9ab")
How it ran

no invocation recorded.

load_tokenizer_for_eval Not yet run
unverified (harvested, no recorded run)
SunGL001/OGPSA/eval/AlpacaEval.py
code served (permissive licence) · get_code("10cae133e44ed5f1")
How it ran

no invocation recorded.

Also present in other papers' code

Of 18 samples lifted from this paper's linked repositories, 18 carry a function identity and 0 do not. Of the 18 distinct functions with an identity, 0 are also present in other papers whose code we hold (0 with at least one paper dated earlier, 0 the same month, 0 later) and 18 are only here among the 52,977 papers whose code Syntology holds (as of 2026-09-26).

Only here means: among the 52,977 papers whose code Syntology holds (51,887 with lifted samples, 1,090 through their assemblies only), as of 2026-09-26, no other paper's lifted samples or current assembly bricks contain an identical function body after normalisation.

Presence, not direction: an identical function body in two papers' repositories does not say who wrote it, which came from which, or that either paper's method rests on it. Matching is exact after normalisation, so a renamed or edited function is not matched. Dates are when papers appeared, not when code was written. Only .py files the harvester lifted are covered; notebooks are not read. A new-style arXiv id YYMM.NNNN dates a paper to the month 20YY-MM; otherwise Paper.year dates it to the year; otherwise it is undated. Dates are when the paper appeared, not when the code was written. Read 2026-11-22.

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

Safety post-training can improve the harmfulness and policy compliance of Large Language Models (LLMs), but it may also reduce general utility, a phenomenon often described as the alignment tax. We study this trade-off through the lens of continual learning: sequential alignment stages expose the model to shifted data distributions and objectives, and their gradients may interfere with directions that support previously acquired general capabilities. This view does not claim that all alignment degradation has a single cause; rather, it provides a useful firstorder mechanism for mitigating one important source of capability regression.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2602.07892")
get_code_for_paper("2602.07892")
have("2602.07892")

The run record, dated, one paper per request, free:

curl https://syntology.ai/api/ran/2602.07892.json

A badge for a README (the split and the date, never a ratio):

[![Syntology run record](https://syntology.ai/api/ran/2602.07892.svg)](https://syntology.ai/paper/2602.07892)

Connect an agent — have() is free.