| build_chat_messages |
Ran ran; our contract draft was wrong, not the code |
Da1yuqin/SEAD/SEAD/vllm_service/vllm_server.py code served (permissive licence) · get_code("96f494ea053bac2d") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned list, ndim 1, dtype family float; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import logging
from typing import List, Dict, Any
logging.basicConfig(level=logging.INFO)
# Set up arguments for build_chat_messages function
base_system_prompt_template = """You are a helpful assistant.
Initial cooperation: ${cooperation}
Initial emotion: ${emotion}
Initial trust: ${trust}
Specific behaviors: ${behaviors}
"""
dialogue_history = [
{
"role": "user",
"content": "<user_reply>Hello, I need help with my account.</user_reply>"
},
{
"role": "assistant",
"content": "I'd be happy to help you with your account. What seems to be the issue?"
},
{
"role": "user",
"content": "<user_reply>I can't reset my password.</user_reply>"
}
]
user_params = {
"cooperation": "high",
"emotion": "neutral",
"trust": "medium",
"behaviors": ["be helpful", "respond quickly", "stay professional"]
}
mode = "call_chatbot"
ARGS = {
"base_system_prompt_template": base_system_prompt_template,
"dialogue_history": dialogue_history,
"user_params": user_params,
"mode": mode
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type list, ndim 1, dtype family float — a list output has no array shape of its own; none is stated
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| calculate_completion_rate |
Ran ran, honoured the contract we drafted |
Da1yuqin/SEAD/utils/analyze_chatbot_mistakes.py code served (permissive licence) · get_code("3c337ab1bef2b5b1") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned float, ndim 0, dtype family float, shape () as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import numpy as np
from typing import List, Dict
torch.manual_seed(0)
# Construct test dialogues
# The function checks if d.get('batch_statuses', []) exists and if d['batch_statuses'][-1] == 1
dialogues = [
{
'batch_statuses': [0, 0, 1] # success_count += 1 (last status is 1)
},
{
'batch_statuses': [1, 0, 0] # success_count += 0 (last status is 0)
},
{
'batch_statuses': [1] # success_count += 1 (last status is 1)
},
{
'id': 'dialogue_4' # success_count += 0 (no batch_statuses key)
},
{
'batch_statuses': [] # success_count += 0 (empty list)
}
]
ARGS = {
'dialogues': dialogues
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type float, ndim 0, dtype family float, shape () — shape () as a later run recorded it
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| check_vllm_service_health |
Ran ran; our contract draft was wrong, not the code |
Da1yuqin/SEAD/for_evaluation/Baseline_test_local_models_vllm.py code served (permissive licence) · get_code("5b7295d80bd7a043") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned tuple. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import numpy as np
from typing import List, Tuple
torch.manual_seed(0)
# The function check_vllm_service_health takes a list of server URLs and a timeout
# It makes HTTP requests to check health status
# We need to provide valid URLs (they don't need to actually work since exceptions are caught)
ARGS = {
"server_urls": [
"http://localhost:8000",
"http://localhost:8001",
"http://127.0.0.1:8002"
],
"timeout": 5
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- numpy, requests, torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type tuple — a tuple output has no array shape of its own; none is stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| extract_initial_state |
Ran ran; our contract draft was wrong, not the code |
Da1yuqin/SEAD/utils/analyze_chatbot_mistakes.py code served (permissive licence) · get_code("b364d075c39bce10") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned dict. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import re
from typing import List, Dict, Tuple
# Create a valid dialogue dict with ground_truth_history
dialogue_with_states = {
'ground_truth_history': [
{
'role': 'system',
'content': 'System message'
},
{
'role': 'user',
'content': 'Hello, I am interested. <user_cooperation>3</user_cooperation> <user_emotion>2</user_emotion> <user_trust>1</user_trust>'
}
]
}
dialogue_empty = {
'ground_truth_history': []
}
dialogue_short = {
'ground_truth_history': [
{
'role': 'system',
'content': 'Only one message'
}
]
}
dialogue_no_tags = {
'ground_truth_history': [
{
'role': 'system',
'content': 'System message'
},
{
'role': 'user',
'content': 'Hello without tags'
}
]
}
ARGS = {
'dialogue': dialogue_with_states
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type dict — a dict output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a dict output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| extract_score |
Ran ran, honoured the contract we drafted |
Da1yuqin/SEAD/utils/analyze_chatbot_mistakes.py code served (permissive licence) · get_code("fb8290dd4e4e685b") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned int, ndim 0, dtype family int, shape () as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import re
import torch
def extract_score(content: str, tag: str) -> int:
"""Extract score from XML tag"""
try:
match = re.search(f'<{tag}[^>]*>(\\d+)</{tag}>', content, re.IGNORECASE)
return int(match.group(1)) if match else None
except:
return None
# Setup code to construct valid arguments
content = "<score>42</score>"
tag = "score"
ARGS = {
"content": content,
"tag": tag
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type int, ndim 0, dtype family int, shape () — shape () as a later run recorded it
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| extract_stages_from_dialogue |
Ran ran, honoured the contract we drafted |
Da1yuqin/SEAD/for_evaluation/get_metrics_from_jsonl.py code served (permissive licence) · get_code("949edfca00534d9f") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned list, ndim 1, dtype family int; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import re
def extract_tag_value(text: str, tag_name: str, cast_type=None):
"""Extract value from XML-style tags"""
pattern = f'<{tag_name}[^>]*>(.*?)</{tag_name}>'
match = re.search(pattern, text, re.DOTALL | re.IGNORECASE)
if match:
value_str = match.group(1).strip()
if cast_type:
try:
return cast_type(value_str)
except (ValueError, TypeError):
return None
return value_str
return None
def extract_stages_from_dialogue(dialogue_item):
"""
Extract all stages that appear in the dialogue
Returns: List[int]
"""
ground_truth_history = dialogue_item.get('ground_truth_history', [])
stages = []
for msg in ground_truth_history:
if msg['role'] == 'assistant':
content = msg['content']
stage_val = extract_tag_value(content, 'stage', int)
if stage_val is not None:
stages.append(stage_val)
return stages
# Construct valid arguments
dialogue_item = {
'ground_truth_history': [
{
'role': 'user',
'content': 'Hello, what is the first stage?'
},
{
'role': 'assistant',
'content': 'The first stage is <stage>1</stage>.'
},
{
'role': 'user',
'content': 'What about the second stage?'
},
{
'role': 'assistant',
'content': 'The second stage is <stage>2</stage> of the process.'
},
{
'role': 'assistant',
'content': 'Then we move to <stage>3</stage>.'
}
]
}
ARGS = {
'dialogue_item': dialogue_item
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type list, ndim 1, dtype family int — a list output has no array shape of its own; none is stated
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| extract_tag_value |
Ran ran; our contract draft was wrong, not the code |
Da1yuqin/SEAD/for_evaluation/get_metrics_from_jsonl.py code served (permissive licence) · get_code("0b7e36b128f91713") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned str. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import re
def extract_tag_value(text: str, tag_name: str, cast_type=None):
"""Extract value from XML-style tags"""
pattern = f'<{tag_name}[^>]*>(.*?)</{tag_name}>'
match = re.search(pattern, text, re.DOTALL | re.IGNORECASE)
if match:
value_str = match.group(1).strip()
if cast_type:
try:
return cast_type(value_str)
except (ValueError, TypeError):
return None
return value_str
return None
ARGS = {
"text": "<greeting>Hello World</greeting>",
"tag_name": "greeting",
"cast_type": None
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type str — a str output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a str output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| extract_user_final_status |
Ran ran, honoured the contract we drafted |
Da1yuqin/SEAD/for_evaluation/get_metrics_from_jsonl.py code served (permissive licence) · get_code("fa77461d2cd2929c") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned int, ndim 0, dtype family int, shape () as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import re
def extract_user_final_status(dialogue_item):
"""Extract user's final status from dialogue"""
ground_truth_history = dialogue_item.get('ground_truth_history', [])
if not ground_truth_history:
return None
for msg in reversed(ground_truth_history):
if msg['role'] == 'user':
content = msg['content']
match = re.search('<user_status>\\s*(-?\\d+)\\s*</user_status>', content, re.IGNORECASE)
if match:
return int(match.group(1))
break
return None
# Setup code
dialogue_item = {
'ground_truth_history': [
{
'role': 'assistant',
'content': 'Hello, how are you?'
},
{
'role': 'user',
'content': 'I am doing well. <user_status> 42 </user_status>'
}
]
}
ARGS = {
'dialogue_item': dialogue_item
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type int, ndim 0, dtype family int, shape () — shape () as a later run recorded it
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| fill_user_params |
Ran ran; our contract draft was wrong, not the code |
Da1yuqin/SEAD/SEAD/vllm_service/vllm_server.py code served (permissive licence) · get_code("c8a9c51bedd5ee37") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned str. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import re
import logging
from typing import Dict, Any
logging.basicConfig(level=logging.INFO)
# Define ARGS with valid arguments for fill_user_params
ARGS = {
"template": """
Please create a character with the following traits:
- Cooperation level: ${cooperation}
- Emotional state: ${emotion}
- Trust level: ${trust}
- Required behaviors: {behaviors}
Additional notes:
Some text with ${initial_cooperation} and {initial_emotion} placeholders.
""",
"user_params": {
"cooperation": "high",
"emotion": "optimistic",
"trust": "moderate",
"behaviors": ["be helpful", "ask clarifying questions", "maintain professionalism"]
}
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type str — a str output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a str output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| fill_user_params |
Ran ran; our contract draft was wrong, not the code |
Da1yuqin/SEAD/for_evaluation/Baseline_test_local_models_vllm.py code served (permissive licence) · get_code("cf0a4a3e934a7c1d") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned str. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import re
from typing import Dict, List, Tuple, Optional, Any
def fill_user_params(template: str, user_params: Dict[str, Any]) -> str:
"""
Fill user_params placeholders.
Only removes ${xxx} format placeholders, not {xxx}.
"""
try:
filled = template
field_mapping = {'cooperation': 'initial_cooperation', 'emotion': 'initial_emotion', 'trust': 'initial_trust', 'behaviors': 'specific_behaviors'}
for key, value in user_params.items():
if key == 'behaviors' and isinstance(value, list):
if value:
value_str = '\n'.join([f'- {b}' for b in value])
else:
value_str = '- No specific behavior requirements'
else:
value_str = str(value)
placeholders_to_try = []
placeholders_to_try.append(f'${{{key}}}')
mapped_key = field_mapping.get(key)
if mapped_key:
placeholders_to_try.append(f'${{{mapped_key}}}')
for placeholder in placeholders_to_try:
if placeholder in filled:
filled = filled.replace(placeholder, value_str)
break
filled = re.sub('\\$\\{[^}]+\\}', '', filled)
return filled
except Exception as e:
print(f'Error filling user_params: {e}')
return template
ARGS = {
'template': 'The user has ${cooperation} cooperation level, ${emotion} emotion, and ${trust} trust level. Behaviors: ${behaviors}. Extra placeholder: ${extra}.',
'user_params': {
'cooperation': 'high',
'emotion': 'positive',
'trust': 'medium',
'behaviors': ['be friendly', 'listen carefully', 'respond promptly']
}
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type str — a str output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a str output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| load_analysis |
Ran ran; our contract draft was wrong, not the code |
Da1yuqin/SEAD/utils/generate_profiles_parallel.py code served (permissive licence) · get_code("d56b142dd29f350c") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned dict. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import json
import os
import tempfile
# Create a temporary directory for testing
temp_dir = tempfile.mkdtemp()
# Create a valid JSON analysis file
analysis_data = {
"model": "test_model",
"accuracy": 0.95,
"metrics": ["precision", "recall", "f1"],
"results": {"train": 0.92, "val": 0.90}
}
analysis_file = os.path.join(temp_dir, "analysis.json")
with open(analysis_file, 'w', encoding='utf-8') as f:
json.dump(analysis_data, f)
ARGS = {
"analysis_file": analysis_file
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type dict — a dict output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a dict output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| load_behavior_library |
Ran ran; our contract draft was wrong, not the code |
Da1yuqin/SEAD/utils/create_prompt_data.py code served (permissive licence) · get_code("afee7667b68ea938") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned dict. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import json
from pathlib import Path
import tempfile
import os
# Create a temporary directory and file for the behavior library
temp_dir = tempfile.mkdtemp()
library_file = Path(temp_dir) / "behavior_library.json"
# Create valid behavior library data
library_data = {
"User_Persona_Classification": {
"category_1": ["action_1", "action_2", "action_3"],
"category_2": ["action_4", "action_5"],
"category_3": ["action_6", "action_7", "action_8", "action_9"]
}
}
# Write the library file
with open(library_file, 'w', encoding='utf-8') as f:
json.dump(library_data, f)
# Define ARGS with the library_file parameter
ARGS = {
"library_file": library_file
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type dict — a dict output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a dict output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| load_behavior_library |
Ran ran; our contract draft was wrong, not the code |
Da1yuqin/SEAD/utils/generate_profiles_parallel.py code served (permissive licence) · get_code("a528e94a3ea5d552") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned dict. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import json
import tempfile
from pathlib import Path
# Create a temporary directory and file for testing
temp_dir = tempfile.mkdtemp()
library_path = Path(temp_dir) / "behavior_library.json"
# Create a sample behavior library JSON file
behavior_data = {
"User_Persona_Classification": {
"category_1": ["action_1", "action_2", "action_3"],
"category_2": ["action_2", "action_4"],
"category_3": ["action_5"]
}
}
with open(library_path, 'w', encoding='utf-8') as f:
json.dump(behavior_data, f)
ARGS = {
"library_file": str(library_path)
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type dict — a dict output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a dict output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| load_profiles_from_file |
Ran ran; our contract draft was wrong, not the code |
Da1yuqin/SEAD/utils/create_prompt_data.py code served (permissive licence) · get_code("475f74719b2c7a8f") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned tuple. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import json
import tempfile
from pathlib import Path
# Create a temporary file with valid profile data
temp_file = tempfile.NamedTemporaryFile(mode='w', suffix='.jsonl', delete=False, encoding='utf-8')
temp_file_path = temp_file.name
# Write sample profiles to the temporary file
profiles_data = [
{'cooperation': 0, 'emotion': 1, 'trust': 2},
{'cooperation': 1, 'emotion': 0, 'trust': 1},
{'cooperation': 0, 'emotion': 1, 'trust': 0},
{'cooperation': 1, 'emotion': 1, 'trust': 2},
{'cooperation': 0, 'emotion': 0, 'trust': 1},
{'cooperation': 1, 'emotion': 2, 'trust': 0},
{'cooperation': 2, 'emotion': 1, 'trust': 2},
{'cooperation': 0, 'emotion': 2, 'trust': 1},
{'cooperation': 1, 'emotion': 0, 'trust': 2},
{'cooperation': 2, 'emotion': 0, 'trust': 0},
]
for profile in profiles_data:
temp_file.write(json.dumps(profile) + '\n')
temp_file.close()
ARGS = {
'file_path': temp_file_path,
'train_samples': 6,
'test_samples': 2,
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type tuple — a tuple output has no array shape of its own; none is stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| load_prompt_file |
Ran ran; our contract draft was wrong, not the code |
Da1yuqin/SEAD/SEAD/vllm_service/vllm_server.py code served (permissive licence) · get_code("c7bcd40345c48edd") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned str. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import os
import tempfile
# Create a temporary directory with prompt files for testing
temp_dir = tempfile.mkdtemp()
# Create sample prompt files
with open(os.path.join(temp_dir, 'call_client.txt'), 'w', encoding='utf-8') as f:
f.write('You are a client support assistant.')
with open(os.path.join(temp_dir, 'call_chatbot.txt'), 'w', encoding='utf-8') as f:
f.write('You are a chatbot assistant.')
with open(os.path.join(temp_dir, 'custom.txt'), 'w', encoding='utf-8') as f:
f.write('You are a custom assistant.')
ARGS = {
'prompt_dir': temp_dir,
'mode': 'call_client'
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type str — a str output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a str output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| read_prompt |
Ran ran; our contract draft was wrong, not the code |
Da1yuqin/SEAD/utils/create_prompt_data.py code served (permissive licence) · get_code("18b5ff46b3ab5402") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned str. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
from pathlib import Path
import tempfile
import os
# Create a temporary directory and file for testing
temp_dir = tempfile.mkdtemp()
temp_file = Path(temp_dir) / "prompt.txt"
temp_file.write_text("This is a sample prompt.\n", encoding='utf-8')
ARGS = {
'file_path': temp_file
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type str — a str output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a str output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| restart_vllm_service |
Ran ran, violated the contract we drafted |
Da1yuqin/SEAD/for_evaluation/Baseline_test_local_models_vllm.py code served (permissive licence) · get_code("8d1e5517fde1b5f5") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned bool, ndim 0, dtype family float, shape () as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import os
import tempfile
# Create a temporary directory for the restart script
temp_dir = tempfile.mkdtemp()
restart_script_path = os.path.join(temp_dir, "start_vllm.sh")
# Create a minimal bash script that does nothing (won't actually run)
with open(restart_script_path, 'w') as f:
f.write("#!/bin/bash\n")
f.write("# Minimal restart script\n")
f.write("exit 0\n")
os.chmod(restart_script_path, 0o755)
# Create a minimal model path
model_path = "/tmp/dummy_model"
# Define ARGS dict with all required parameters
ARGS = {
"restart_script": restart_script_path,
"model_path": model_path,
"gpu_ids": "0,1",
"mode": "call_client",
"max_wait_time": 5
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- psutil, requests — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type bool, ndim 0, dtype family float, shape () — shape () as a later run recorded it
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| sample_random_profile |
Ran ran; our contract draft was wrong, not the code |
Da1yuqin/SEAD/utils/generate_profiles_parallel.py code served (permissive licence) · get_code("52e90609827ad734") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned tuple. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import random
from typing import Dict, List, Tuple, Set
# Create a behavior library with some sample actions
behavior_lib = {
'all_actions': [
'cooperate',
'defect',
'share',
'punish',
'reward',
'negotiate',
'compromise',
'escalate'
]
}
# Create a random number generator with a fixed seed for reproducibility
rng = random.Random(42)
# Create an analysis dict with recommended weights
analysis = {
'recommended_weights': {
'cooperation_weights': [0, 0.33, 0.34, 0.33, 0],
'emotion_weights': [0, 0.5, 0.5, 0],
'trust_weights': [0, 0.25, 0.25, 0.25, 0.25, 0]
}
}
# Set up the arguments dictionary
ARGS = {
'behavior_lib': behavior_lib,
'rng': rng,
'analysis': analysis,
'min_behaviors': 1,
'max_behaviors': 3
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type tuple — a tuple output has no array shape of its own; none is stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| unpad_dataproto |
Ran ran; our contract draft was wrong, not the code |
Da1yuqin/SEAD/verl/protocol.py code served (permissive licence) · get_code("25f0ea3f460f6ce1") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1) and returned DataProto. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import numpy as np
torch.manual_seed(0)
class DataProto:
"""Stand-in class for DataProto that supports slicing."""
def __init__(self, data):
self.data = data
def __getitem__(self, key):
"""Support slicing to return a new DataProto with sliced data."""
sliced_data = self.data[key]
return DataProto(sliced_data)
def __repr__(self):
return f"DataProto({self.data})"
# Create a small DataProto instance with some data
data_tensor = torch.randn(2, 4)
data_proto = DataProto(data_tensor)
ARGS = {
"data": data_proto,
"pad_size": 0
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1
- Imports
- numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type DataProto — a DataProto output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a DataProto output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| get_weight_loader |
Not yet run unverified (harvested, no recorded run) |
Da1yuqin/SEAD/verl/models/weight_loader_registry.py code served (permissive licence) · get_code("919cf310ad4c189a") How it ran |