SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2109.14739 · 2021

Multi-Task Pre-Training for Plug-and-Play Task-Oriented Dialogue System

arXiv · PDF · Open in the Atlas

Code that ran

10 of 10 samples (functions and classes) lifted from this paper's linked repositories executed on a synthesized input in Syntology's jail (0 of them constructed an object rather than computing a result; 0 honoured the contract we drafted, 0 violated it and 9 have no contract verdict; 1 failure of our instrument; 0 no recorded run), measured 2026-11-22; repository links from the Papers with Code archive attributed under CC BY-SA, no affiliation; bodies inline only under a permissive licence as GitHub detects it. Not a reproduction of the paper's results.

"Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results. This record, dated, as JSON: /api/ran/2109.14739.json · badge: /api/ran/2109.14739.svg

Each repository's role is a label: a classifier's output recorded on the link, or, where no role is recorded, what the Papers with Code archive marked. The line under it names its basis. Neither says who wrote the code.

RepositoryRoleRan
awslabs/pptod canonical (classifier label)
classifier basis not recorded on the link
10 ran · 0 no recorded run
FunctionStatusWhere it lives
batch_generate Ran
ran on a synthesized input
awslabs/pptod/DST/inference_utlis.py
code served (permissive licence) · get_code("157e7d21e425e71b")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned list, ndim 1, dtype family float; no output shape is stated. The image it ran in was not recorded. The code mentions cuda, and the harness aliases cuda to the CPU for every sample; whether this run needed that is not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn

torch.manual_seed(0)

# Create a minimal model that satisfies the function's usage
class MinimalModel(nn.Module):
    def __init__(self):
        super().__init__()
        self.linear = nn.Linear(4, 4)
    
    def batch_generate(self, bs_tensor, bs_mask, generate_mode=None, max_decode_len=None):
        # Return a list of generated text strings, one per batch item
        batch_size = bs_tensor.shape[0]
        return [f"generated_text_{i}" for i in range(batch_size)]

# Create a minimal tokenizer
class MinimalTokenizer:
    def encode(self, text):
        return [1, 2, 3]
    
    def decode(self, tokens):
        return "decoded_text"

# Create a minimal data object
class MinimalData:
    def __init__(self):
        self.tokenizer = MinimalTokenizer()
    
    def pad_batch(self, batch):
        # Return padded tensor and mask
        batch_size = len(batch)
        # bs_tensor: shape (batch_size, seq_len)
        # bs_mask: shape (batch_size, seq_len)
        bs_tensor = torch.randn(batch_size, 4)
        bs_mask = torch.ones(batch_size, 4)
        return bs_tensor, bs_mask

# Create model and move to CPU (not CUDA)
model = MinimalModel()

# Create one_inference_batch: (bs_batch, parse_dict_batch)
bs_batch = [["text_sample_0"], ["text_sample_1"]]  # 2 samples
parse_dict_batch = [
    {"bspn": "belief_state_0"},
    {"bspn": "belief_state_1"}
]
one_inference_batch = (bs_batch, parse_dict_batch)

# Create data object
data = MinimalData()

ARGS = {
    "model": model,
    "one_inference_batch": one_inference_batch,
    "data": data
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type list, ndim 1, dtype family float — a list output has no array shape of its own; none is stated
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
clean_text Ran
ran on a synthesized input
awslabs/pptod/E2E_TOD/clean_dataset.py
code served (permissive licence) · get_code("ea3e5b726884db6d")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned str. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import tempfile
import os

# Create a temporary mapping file for the clean_text function
temp_dir = tempfile.mkdtemp()
mapping_file = os.path.join(temp_dir, 'mapping.txt')

# Write some sample mappings to the file
with open(mapping_file, 'w') as f:
    f.write('hello\thello there\n')
    f.write('world\tuniverse\n')
    f.write('test\ttesting\n')

ARGS = {
    'text': "Don't forget at 3pm, call me at 01223358966. It's a b&b with a swimming pool. Departure: 2:30am.",
    'mapping_pair_path': mapping_file
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type str — a str output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a str output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
clean_time Ran
ran; our contract draft was wrong, not the code
awslabs/pptod/E2E_TOD/clean_dataset.py
code served (permissive licence) · get_code("4c62f23ce8135bda")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned str. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import re

def clean_time(utter):
    utter = re.sub('(\\d+) ([ap]\\.?m)', lambda x: x.group(1) + x.group(2), utter)
    utter = re.sub('((?<!\\d)\\d:\\d+)(am)?', '0\\1', utter)
    utter = re.sub('((?<!\\d)\\d)am', '0\\1:00', utter)
    utter = re.sub('((?<!\\d)\\d)pm', lambda x: str(int(x.group(1)) + 12) + ':00', utter)
    utter = re.sub('(\\d+)(:\\d+)pm', lambda x: str(int(x.group(1)) + 12) + x.group(2), utter)
    utter = re.sub('(\\d+)a\\.?m', '\\1', utter)
    return utter

ARGS = {
    'utter': 'The meeting is at 3 p.m. tomorrow at 2:30am'
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type str — a str output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a str output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
erase_error Ran
ran on a synthesized input
awslabs/pptod/DST/inference_utlis.py
code served (permissive licence) · get_code("ddc6b154895626c0")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned str. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import re

def erase_error(text):
    text = re.sub('\\[value -', '[value\\-', text)
    text = re.sub('\\[ value', '[value', text)
    text = re.sub('\\[value-', '[value_', text)
    return text

ARGS = {
    "text": "[value - error] and [ value test] with [value-case]"
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type str — a str output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a str output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
fix_mismatch_jason Ran
ran on a synthesized input
awslabs/pptod/DST/dst.py
code served (permissive licence) · get_code("0ee004932e081b43")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned tuple, ndim 1, dtype family float; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

# Analyze the function signature
# fix_mismatch_jason(slot, value)
# The function takes two parameters: slot (string) and value (string)
# It performs string comparisons and returns a tuple of (slot, value)

# Construct valid arguments
ARGS = {
    "slot": "type",
    "value": "nigh"
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type tuple, ndim 1, dtype family float — a tuple output has no array shape of its own; none is stated
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
get_checkpoint_name Ran
ran on a synthesized input
awslabs/pptod/E2E_TOD/inference_pptod.py
code served (permissive licence) · get_code("bb26485d8e62dade")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned str. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import os
import tempfile
import shutil

# Create a temporary directory with a checkpoint file
temp_dir = tempfile.mkdtemp()
checkpoint_file = os.path.join(temp_dir, 'epoch_5.pt')
with open(checkpoint_file, 'w') as f:
    f.write('dummy checkpoint')

ARGS = {
    'prefix': temp_dir
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type str — a str output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a str output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
ignore_none Ran
ran on a synthesized input
awslabs/pptod/DST/dst.py
code served (permissive licence) · get_code("213ac11a76f93ae8")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned tuple, ndim 2, dtype family float; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch

torch.manual_seed(0)

# Construct valid arguments for ignore_none function
# The function takes pred_belief and target_belief as parameters
# Both should be lists of strings representing belief states

pred_belief = [
    "hotel name expensive",
    "restaurant food italian",
    "not mentioned",
    "catherine s restaurant",
    "none",
    "dontcare hotel area"
]

target_belief = [
    "hotel name moderate",
    "restaurant food french",
    "not mentioned",
    "dontcare restaurant pricerange",
    "none",
    "attraction type museum"
]

ARGS = {
    "pred_belief": pred_belief,
    "target_belief": target_belief
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type tuple, ndim 2, dtype family float — a tuple output has no array shape of its own; none is stated
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
my_clean_text Ran
ran on a synthesized input
awslabs/pptod/E2E_TOD/clean_dataset.py
code served (permissive licence) · get_code("7f5a7dafc8e4429c")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned str. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import re

def my_clean_text(text):
    text = re.sub('([a-zT]+)\\.([a-z])', '\\1 . \\2', text)
    text = re.sub('(\\w+)\\.\\.? ', '\\1 . ', text)
    return text

ARGS = {
    'text': 'Hello.world. This is a test.. More text.T.here'
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type str — a str output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a str output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
restore_text Ran
ran on a synthesized input
awslabs/pptod/DST/inference_utlis.py
code served (permissive licence) · get_code("5d59ec0edc0482ac")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned str. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import re

def restore_text(text, mode):
    if mode == 'bs':
        text = re.sub(' is ', ' ', text)
    elif mode == 'da':
        pass
    else:
        raise Exception('Wrong Restore Mode!!!')
    text = re.sub(' , ', ' ', text)
    text = ' '.join(text.split()).strip()
    return text


ARGS = {
    'text': 'This is a test , with some spaces',
    'mode': 'bs'
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type str — a str output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a str output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
zip_result Ran
ran on a synthesized input
awslabs/pptod/DST/inference.py
code served (permissive licence) · get_code("79bc40569254969f")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned dict. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import numpy as np

torch.manual_seed(0)

# Construct a valid prediction argument
# The function expects an iterable of dictionaries, where each dict has:
# - 'dial_id': a dialogue identifier
# - 'turn_num': a turn number
# - potentially other keys

prediction = [
    {
        'dial_id': 'dial_001',
        'turn_num': 0,
        'response': 'Hello',
        'confidence': 0.95
    },
    {
        'dial_id': 'dial_001',
        'turn_num': 1,
        'response': 'How are you?',
        'confidence': 0.87
    },
    {
        'dial_id': 'dial_002',
        'turn_num': 0,
        'response': 'Hi there',
        'confidence': 0.92
    },
    {
        'dial_id': 'dial_002',
        'turn_num': 1,
        'response': 'Nice to meet you',
        'confidence': 0.88
    }
]

ARGS = {
    'prediction': prediction
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type dict — a dict output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a dict output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block

Also present in other papers' code

Of 10 samples lifted from this paper's linked repositories, 10 carry a function identity and 0 do not. Of the 10 distinct functions with an identity, 4 are also present in other papers whose code we hold (1 with at least one paper dated earlier, 0 the same month, 4 later) and 6 are only here among the 52,977 papers whose code Syntology holds (as of 2026-09-26).

Only here means: among the 52,977 papers whose code Syntology holds (51,887 with lifted samples, 1,090 through their assemblies only), as of 2026-09-26, no other paper's lifted samples or current assembly bricks contain an identical function body after normalisation.

FunctionAlso present inEarliest-datedOthers named
clean_time 7 other papers
1 dated earlier, 5 later, 1 same year, order not determinable
2009.12005 (2020-09)
MinTL: Minimalist Transfer Learning for Task-Oriented Dialogue Systems
2021.findings-emnlp.112 (2021 (year only))
Improving End-to-End Task-Oriented Dialogue System with A Simple Auxiliary Task
2204.00763 (2022-04)
Metaphorical User Simulators for Evaluating Task-oriented Dialogue Systems
and 4 more not named here
my_clean_text 3 other papers
3 later
2210.08692 (2022-10)
A Generative User Simulator with GPT-based Architecture and Goal State Tracking for Reinforced Multi-Domain Dialog Systems
2302.11520 (2023-02)
Guiding Large Language Models via Directional Stimulus Prompting
2305.02468 (2023-05)
Task-Optimized Adapters for an End-to-End Task-Oriented Dialogue System
clean_text 1 other paper
1 later
2302.11520 (2023-02)
Guiding Large Language Models via Directional Stimulus Prompting
restore_text 1 other paper
1 later
2305.02468 (2023-05)
Task-Optimized Adapters for an End-to-End Task-Oriented Dialogue System

Presence, not direction: an identical function body in two papers' repositories does not say who wrote it, which came from which, or that either paper's method rests on it. Matching is exact after normalisation, so a renamed or edited function is not matched. Dates are when papers appeared, not when code was written. Only .py files the harvester lifted are covered; notebooks are not read. A new-style arXiv id YYMM.NNNN dates a paper to the month 20YY-MM; otherwise Paper.year dates it to the year; otherwise it is undated. Dates are when the paper appeared, not when the code was written. Read 2026-11-22.

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

Pre-trained language models have been recently shown to benefit task-oriented dialogue (TOD) systems. Despite their success, existing methods often formulate this task as a cascaded generation problem which can lead to error accumulation across different sub-tasks and greater data annotation overhead. In this study, we present PPTOD, a unified plug-and-play model for task-oriented dialogue. In addition, we introduce a new dialogue multi-task pre-training strategy that allows the model to learn the primary TOD task completion skills from heterogeneous dialog corpora. We extensively test our model on three benchmark TOD tasks, including end-to-end dialogue modelling, dialogue state tracking, and intent classification. Experimental results show that PPTOD achieves new state of the art on all evaluated tasks in both high-resource and low-resource scenarios. Furthermore, comparisons against previous SOTA methods show that the responses generated by PPTOD are more factually correct and semantically coherent as judged by human annotators.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2109.14739")
get_code_for_paper("2109.14739")
have("2109.14739")

The run record, dated, one paper per request, free:

curl https://syntology.ai/api/ran/2109.14739.json

A badge for a README (the split and the date, never a ratio):

[![Syntology run record](https://syntology.ai/api/ran/2109.14739.svg)](https://syntology.ai/paper/2109.14739)

Connect an agent — have() is free.