SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2402.08702 · EMNLP · 2024

PRompt Optimization in Multi-Step Tasks (PROMST): Integrating Human Feedback and Heuristic-based Sampling

Yang Zhang, Chuchu Fan, Nicholas Roy, Yongchao Chen, Jacob Arkin, Yilun Hao

arXiv · PDF · Open in the Atlas

Code that ran

9 of 12 samples (functions and classes) lifted from this paper's linked repositories executed on a synthesized input in Syntology's jail (0 of them constructed an object rather than computing a result; 2 honoured the contract we drafted, 0 violated it and 0 have no contract verdict; 7 failures of our instrument; 3 no recorded run), measured 2026-11-22; repository links from the Papers with Code archive attributed under CC BY-SA, no affiliation; bodies inline only under a permissive licence as GitHub detects it. Not a reproduction of the paper's results.

"Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results. This record, dated, as JSON: /api/ran/2402.08702.json · badge: /api/ran/2402.08702.svg

Each repository's role is a label: a classifier's output recorded on the link, or, where no role is recorded, what the Papers with Code archive marked. The line under it names its basis. Neither says who wrote the code.

RepositoryRoleRan
yongchao98/promst canonical (classifier label)
classifier basis: census
9 ran · 3 no recorded run
FunctionStatusWhere it lives
action_from_response Ran
ran; our contract draft was wrong, not the code
yongchao98/promst/BoxNet1/env1_create.py
code served (permissive licence) · get_code("7436a57078e7696c")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned tuple, ndim 1, dtype family float; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import re
import copy
import numpy as np

# Analyze the function to understand what it needs:
# 1. pg_dict_input: A dict with string keys like "0.0_1.0" mapping to lists of items
# 2. original_response_dict: A dict with string keys containing coordinates and values with "move(...)" patterns

# Build pg_dict_input - maps coordinate strings to lists of items
pg_dict_input = {
    "0.0_1.0": ["box_a", "target_a"],
    "1.0_1.0": ["box_b"],
    "0.0_2.0": ["target_b"],
    "1.0_2.0": [],
    "2.0_1.0": ["box_c", "target_c"],
}

# Build original_response_dict
# Keys should contain coordinates (as strings with numbers)
# Values should have move(item, location) format
original_response_dict = {
    "(0.0, 1.0)": "move(box_a, square(0.0, 2.0))",
    "(1.0, 1.0)": "move(box_b, square(1.0, 2.0))",
    "(2.0, 1.0)": "move(box_c, target_c)",
}

ARGS = {
    "pg_dict_input": pg_dict_input,
    "original_response_dict": original_response_dict,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
numpy — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type tuple, ndim 1, dtype family float — a tuple output has no array shape of its own; none is stated
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
corner_position Ran
ran, honoured the contract we drafted
yongchao98/promst/BoxNet2/env2_create.py
code served (permissive licence) · get_code("f697b88d2cfd7249")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned list, ndim 2, dtype family float; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch

torch.manual_seed(0)

ARGS = {
    "pg_row_i": 5,
    "pg_column_j": 3
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type list, ndim 2, dtype family float — a list output has no array shape of its own; none is stated
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
error_string_func_APO Ran
ran; our contract draft was wrong, not the code
yongchao98/promst/BoxNet1/prompt_env1.py
code served (permissive licence) · get_code("c42105f069a2cdbd")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned str. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import numpy as np

torch.manual_seed(0)

ARGS = {
    'rear_prompt_1': 'Current state: robot at position (0, 0). Task: navigate to goal (5, 5).',
    'response_1': 'Plan: Move right 5 steps, then move up 5 steps to reach the goal.',
    'env_act_feedback_1': 'Executed move right 3 steps. New position: (3, 0). Obstacle detected ahead.',
    'rear_prompt_2': 'Current state: robot at position (3, 0). Previous action partially blocked. Task: navigate to goal (5, 5).',
    'response_2': 'Revised plan: Move around obstacle by going up 2 steps, then right 2 steps, then up 3 steps.'
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type str — a str output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a str output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
judge_move_box2pos_box2target_func Ran
ran; our contract draft was wrong, not the code
yongchao98/promst/BoxNet2/env2_create.py
code served (permissive licence) · get_code("53a8de84b6bc929e")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned tuple, ndim 1, dtype family float; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import numpy as np

# Build pg_dict_original
# The function checks for keys like "0.5_0.5", "0.5_1.5", "1.5_0.5", "1.5_1.5"
# and looks up values from those keys
pg_dict_original = {
    "0.5_0.5": ["box_1", "agent_a"],
    "0.5_1.5": ["box_2"],
    "1.5_0.5": ["box_3"],
    "1.5_1.5": ["target_1"],
    "1.0_1.0": ["target_1", "agent_b"],
}

# key should be a tuple where both elements are x.5 (half-integer)
# The function checks np.mod(key[0], 1) == 0.5 and np.mod(key[1], 1) == 0.5
key = (1.0, 1.0)

# value is a tuple or similar structure
# value[0] should be a box identifier like "box_1"
# value[1] can be either:
#   - a tuple representing a position
#   - a string representing a target like "target_1"
value = ("box_1", "target_1")

ARGS = {
    "key": key,
    "value": value,
    "pg_dict_original": pg_dict_original,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
numpy — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type tuple, ndim 1, dtype family float — a tuple output has no array shape of its own; none is stated
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
prompt_to_promptLLM_func_total_APE Ran
ran; our contract draft was wrong, not the code
yongchao98/promst/BoxNet1/prompt_env1.py
code served (permissive licence) · get_code("d45869657d4415b9")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned str. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn

# Construct arguments for prompt_to_promptLLM_func_total_APE
prompt_task_explain = "You are a helpful assistant that answers questions about machine learning and deep learning concepts. Please provide clear and concise explanations with examples when possible."

ARGS = {
    "prompt_task_explain": prompt_task_explain
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type str — a str output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a str output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
read_file_content Ran
ran; our contract draft was wrong, not the code
yongchao98/promst/BlocksWorld/env7-blocksworld-list-score.py
code served (permissive licence) · get_code("6c1cdb58b0bb57ab")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned str. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import tempfile
import os

# Create a temporary file with some content
temp_file = tempfile.NamedTemporaryFile(mode='w', delete=False, suffix='.txt')
temp_file.write("Hello World\n  \n")
temp_file.close()

# Store the file path
file_path = temp_file.name

# Define arguments for read_file_content
ARGS = {
    'file_path': file_path
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type str — a str output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a str output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
state_update_func Ran
ran; our contract draft was wrong, not the code
yongchao98/promst/BoxNet1/env1_create.py
code served (permissive licence) · get_code("cebcc1ecc15d62ae")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned str. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import copy

def surround_index_func(row_num, coloum_num, row_index, coloum_index):
    surround_index_list = []
    for i, j in ([row_index - 1, coloum_index], [row_index + 1, coloum_index], [row_index, coloum_index - 1], [row_index, coloum_index + 1]):
        if i >= 0 and i <= row_num - 1 and (j >= 0) and (j <= coloum_num - 1) and (not (i == row_index and j == coloum_index)):
            surround_index_list.append([i + 0.5, j + 0.5])
    return surround_index_list

def state_update_func(pg_row_num, pg_column_num, pg_dict):
    pg_dict_copy = copy.deepcopy(pg_dict)
    state_update_prompt = ''
    for i in range(pg_row_num):
        for j in range(pg_column_num):
            square_item_list = pg_dict_copy[str(i + 0.5) + '_' + str(j + 0.5)]
            square_item_only_box = [item for item in square_item_list if item[:3] == 'box']
            surround_index_list = surround_index_func(pg_row_num, pg_column_num, i, j)
            state_update_prompt += f'Agent[{i + 0.5}, {j + 0.5}]: I am in square[{i + 0.5}, {j + 0.5}], I can observe {square_item_list}, I can do '
            action_list = []
            for box in square_item_only_box:
                for surround_index in surround_index_list:
                    action_list.append(f'move({box}, square{surround_index})')
                if 'target' + box[3:] in square_item_list:
                    action_list.append(f'move({box}, target{box[3:]})')
            state_update_prompt += f'{action_list}\n'
    return state_update_prompt


pg_row_num = 2
pg_column_num = 3

pg_dict = {
    '0.5_0.5': ['box_1', 'target_1', 'wall'],
    '0.5_1.5': ['box_2'],
    '0.5_2.5': [],
    '1.5_0.5': ['target_2'],
    '1.5_1.5': ['box_3', 'target_3'],
    '1.5_2.5': ['wall']
}

ARGS = {
    'pg_row_num': pg_row_num,
    'pg_column_num': pg_column_num,
    'pg_dict': pg_dict
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type str — a str output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a str output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
state_update_func Ran
ran; our contract draft was wrong, not the code
yongchao98/promst/BoxNet2/env2_create.py
code served (permissive licence) · get_code("a683c02bc4f7bac7")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned str. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import copy

# Analyze the function to understand what pg_dict structure is needed
# The function accesses:
# - pg_dict[str(i + 0.5) + '_' + str(j + 0.5)] which should be a list
# - pg_dict[str(corner_x) + '_' + str(corner_y)] which should be a list
# - Items in those lists that look like strings starting with 'box' or 'target'

# Build a minimal pg_dict that satisfies the function's access patterns
pg_dict = {}

# For a 2x2 grid (pg_row_num=2, pg_column_num=2)
# We need center positions (i+0.5, j+0.5) and corner positions (integers)
for i in range(2):
    for j in range(2):
        # Center of each square
        center_key = str(i + 0.5) + '_' + str(j + 0.5)
        pg_dict[center_key] = ['item1', 'target1']
        
        # Corners of each square
        for corner_x, corner_y in [(float(i), float(j)), (float(i), float(j + 1)), 
                                     (float(i + 1), float(j)), (float(i + 1), float(j + 1))]:
            corner_key = str(corner_x) + '_' + str(corner_y)
            if corner_key not in pg_dict:
                # Some corners have boxes, some are empty
                if (corner_x, corner_y) == (0.0, 0.0):
                    pg_dict[corner_key] = ['box1']
                else:
                    pg_dict[corner_key] = []

ARGS = {
    'pg_row_num': 2,
    'pg_column_num': 2,
    'pg_dict': pg_dict
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type str — a str output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a str output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
surround_index_func Ran
ran, honoured the contract we drafted
yongchao98/promst/BoxNet1/env1_create.py
code served (permissive licence) · get_code("6a024a82611e53a1")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned list, ndim 2, dtype family float; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import numpy as np

# This function finds the surrounding indices (up, down, left, right neighbors)
# of a given position in a 2D grid, with boundary checking.
# It takes:
# - row_num: total number of rows in the grid
# - coloum_num: total number of columns in the grid
# - row_index: the row index of the current position
# - coloum_index: the column index of the current position

# Set up valid arguments for the function
ARGS = {
    "row_num": 5,           # 5 rows in the grid
    "coloum_num": 4,        # 4 columns in the grid
    "row_index": 2,         # query position at row 2
    "coloum_index": 1       # query position at column 1
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type list, ndim 2, dtype family float — a list output has no array shape of its own; none is stated
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
GPT_response Not yet run
unverified (harvested, no recorded run)
yongchao98/promst/BoxNet1/LLM.py
code served (permissive licence) · get_code("7f59d2db424b7b8d")
How it ran

no invocation recorded.

score_in_training_set Not yet run
unverified (harvested, no recorded run)
yongchao98/promst/BoxNet1/env1_func.py
code served (permissive licence) · get_code("02b8bef5328aa45d")
How it ran

no invocation recorded.

total_score_in_training_set Not yet run
unverified (harvested, no recorded run)
yongchao98/promst/BoxNet1/env1_func.py
code served (permissive licence) · get_code("c87152fe97b7d199")
How it ran

no invocation recorded.

Also present in other papers' code

Of 12 samples lifted from this paper's linked repositories, 12 carry a function identity and 0 do not. Of the 12 distinct functions with an identity, 0 are also present in other papers whose code we hold (0 with at least one paper dated earlier, 0 the same month, 0 later) and 12 are only here among the 52,977 papers whose code Syntology holds (as of 2026-09-26).

Only here means: among the 52,977 papers whose code Syntology holds (51,887 with lifted samples, 1,090 through their assemblies only), as of 2026-09-26, no other paper's lifted samples or current assembly bricks contain an identical function body after normalisation.

Presence, not direction: an identical function body in two papers' repositories does not say who wrote it, which came from which, or that either paper's method rests on it. Matching is exact after normalisation, so a renamed or edited function is not matched. Dates are when papers appeared, not when code was written. Only .py files the harvester lifted are covered; notebooks are not read. A new-style arXiv id YYMM.NNNN dates a paper to the month 20YY-MM; otherwise Paper.year dates it to the year; otherwise it is undated. Dates are when the paper appeared, not when the code was written. Read 2026-11-22.

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

Prompt optimization aims to find the best prompt to a large language model (LLM) for a given task. LLMs have been successfully used to help find and improve prompt candidates for single-step tasks. However, realistic tasks for agents are multi-step and introduce new challenges: (1) Prompt content is likely to be more extensive and complex, making it more difficult for LLMs to analyze errors, (2) the impact of an individual step is difficult to evaluate, and (3) different people may have varied preferences about task execution. While humans struggle to optimize prompts, they are good at providing feedback about LLM outputs; we therefore introduce a new LLM-driven discrete prompt optimization framework PRompt Optimization in Multi-Step Tasks (PROMST) that incorporates humandesigned feedback rules to automatically offer direct suggestions for improvement. We also use an extra learned heuristic model that predicts prompt performance to efficiently sample from prompt candidates. This approach significantly outperforms both human-engineered prompts and several other prompt optimization methods across 11 representative multi-step tasks (an average 10.6%-29.3% improvement to current best methods on five LLMs respectively). We believe our work can serve as a benchmark for automatic prompt optimization for LLM-driven multi-step tasks.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2402.08702")
get_code_for_paper("2402.08702")
have("2402.08702")

The run record, dated, one paper per request, free:

curl https://syntology.ai/api/ran/2402.08702.json

A badge for a README (the split and the date, never a ratio):

[![Syntology run record](https://syntology.ai/api/ran/2402.08702.svg)](https://syntology.ai/paper/2402.08702)

Connect an agent — have() is free.