SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2106.06169 · ACL · 2021

BoB: BERT Over BERT for Training Persona-based Dialogue Models from Limited Personalized Data

Yan Wang, Ting Liu, Haoyu Song, Kaiyan Zhang, Wei-Nan Zhang

arXiv · PDF · Open in the Atlas

Code that ran

8 of 12 samples (functions and classes) lifted from this paper's linked repositories executed on a synthesized input in Syntology's jail (0 of them constructed an object rather than computing a result; 0 honoured the contract we drafted, 1 violated it and 4 have no contract verdict; 3 failures of our instrument; 4 no recorded run), measured 2026-11-22; bodies inline only under a permissive licence as GitHub detects it. Not a reproduction of the paper's results.

"Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results. This record, dated, as JSON: /api/ran/2106.06169.json · badge: /api/ran/2106.06169.svg

Each repository's role is a label: a classifier's output recorded on the link, or, where no role is recorded, what the Papers with Code archive marked. The line under it names its basis. Neither says who wrote the code.

RepositoryRoleRan
songhaoyu/BoB canonical (classifier label)
classifier basis: census
8 ran · 4 no recorded run
FunctionStatusWhere it lives
invert_mask Ran
ran, violated the contract we drafted
songhaoyu/BoB/xlibs/modeling_bart.py
pointer only (licence: Apache-2.0); the stored copy get_code serves, from yeliu918/KG-BART, is pointer only (licence: NONE) · get_code("874a42377fa0659f")
How it ran

The body of this sample is withheld: this paper's link allows the body inline (Apache-2.0), but get_code serves the stored copy, first harvested from yeliu918/KG-BART, and GitHub detected no licence in that repository when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

make_padding_mask Ran
ran; our contract draft was wrong, not the code
songhaoyu/BoB/xlibs/modeling_bart.py
pointer only (licence: Apache-2.0); the stored copy get_code serves, from yeliu918/KG-BART, is pointer only (licence: NONE) · get_code("f20c9b1aead27fea")
How it ran

The body of this sample is withheld: this paper's link allows the body inline (Apache-2.0), but get_code serves the stored copy, first harvested from yeliu918/KG-BART, and GitHub detected no licence in that repository when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

pad_sequence Ran
ran; our contract draft was wrong, not the code
songhaoyu/BoB/evaluations.py
code served (permissive licence) · get_code("82c47c8c6308eca1")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned chain. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import numpy as np
from itertools import chain

torch.manual_seed(0)

# The function takes:
# - sequence: an iterable of items to pad
# - n: an integer specifying the n-gram size (used to determine padding amount)
# - pad_left: boolean flag
# - pad_right: boolean flag
# - left_pad_symbol: symbol to use for left padding
# - right_pad_symbol: symbol to use for right padding

# Construct a simple sequence (list of tokens/strings)
sequence = ['the', 'cat', 'sat', 'on', 'the', 'mat']

# n parameter - determines how many pad symbols to add (n-1)
n = 3

# Padding flags
pad_left = True
pad_right = True

# Pad symbols
left_pad_symbol = '<START>'
right_pad_symbol = '<END>'

ARGS = {
    'sequence': sequence,
    'n': n,
    'pad_left': pad_left,
    'pad_right': pad_right,
    'left_pad_symbol': left_pad_symbol,
    'right_pad_symbol': right_pad_symbol
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type chain — a chain output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a chain output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
prepare_data_batch Ran
ran on a synthesized input
songhaoyu/BoB/bertoverbert.py
code served (permissive licence) · get_code("a4729bdc1b94786a")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned tuple. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch

torch.manual_seed(0)

# Create a batch dict with the structure expected by prepare_data_batch
batch = {
    'persona': {
        'input_ids': torch.tensor([[101, 2054, 2003, 102], [101, 2054, 2003, 102]]),
        'attention_mask': torch.tensor([[1, 1, 1, 1], [1, 1, 1, 1]]),
        'token_type_ids': torch.tensor([[0, 0, 0, 0], [0, 0, 0, 0]])
    },
    'query': {
        'input_ids': torch.tensor([[101, 2054, 102, 0], [101, 2054, 102, 0]]),
        'attention_mask': torch.tensor([[1, 1, 1, 0], [1, 1, 1, 0]]),
        'token_type_ids': torch.tensor([[0, 0, 0, 0], [0, 0, 0, 0]])
    },
    'response': {
        'input_ids': torch.tensor([[101, 2054, 2003, 102], [101, 2054, 2003, 102]]),
        'attention_mask': torch.tensor([[1, 1, 1, 1], [1, 1, 1, 1]])
    }
}

ARGS = {
    'batch': batch
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type tuple — a tuple output has no array shape of its own; none is stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
prepare_inference_batch Ran
ran on a synthesized input
songhaoyu/BoB/bertoverbert.py
code served (permissive licence) · get_code("771312b20c293c01")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned tuple, ndim 3, dtype family int; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch

torch.manual_seed(0)

# Create sample batch structure with nested dicts containing tokenizer outputs
pos_batch = {
    'pre': {
        'input_ids': torch.tensor([[101, 2054, 2003, 102], [101, 1045, 2572, 102]]),
        'attention_mask': torch.tensor([[1, 1, 1, 1], [1, 1, 1, 1]]),
        'token_type_ids': torch.tensor([[0, 0, 0, 0], [0, 0, 0, 0]])
    },
    'hyp': {
        'input_ids': torch.tensor([[101, 2054, 102, 0], [101, 1045, 102, 0]]),
        'attention_mask': torch.tensor([[1, 1, 1, 0], [1, 1, 1, 0]]),
        'token_type_ids': torch.tensor([[1, 1, 1, 0], [1, 1, 1, 0]])
    }
}

neg_batch = {
    'pre': {
        'input_ids': torch.tensor([[101, 2000, 2003, 102], [101, 2065, 2572, 102]]),
        'attention_mask': torch.tensor([[1, 1, 1, 1], [1, 1, 1, 1]]),
        'token_type_ids': torch.tensor([[0, 0, 0, 0], [0, 0, 0, 0]])
    },
    'hyp': {
        'input_ids': torch.tensor([[101, 2000, 102, 0], [101, 2065, 102, 0]]),
        'attention_mask': torch.tensor([[1, 1, 1, 0], [1, 1, 1, 0]]),
        'token_type_ids': torch.tensor([[1, 1, 1, 0], [1, 1, 1, 0]])
    }
}

ARGS = {
    'pos_batch': pos_batch,
    'neg_batch': neg_batch
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type tuple, ndim 3, dtype family int — a tuple output has no array shape of its own; none is stated
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
read_convai2_split Ran
ran on a synthesized input
songhaoyu/BoB/dataloader.py
code served (permissive licence) · get_code("9a18389a1aeae94a")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned tuple, ndim 2, dtype family float; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import tempfile
import os

# Create a temporary file with ConvAI2 format data
temp_file = tempfile.NamedTemporaryFile(mode='w', delete=False, encoding='utf-8', suffix='.txt')
temp_file_path = temp_file.name

# Write sample ConvAI2 data
sample_data = """your persona: i am a dog lover
your persona: i like to play fetch
1 hello there	hi how are you
your persona: i enjoy reading books
your persona: i live in the city
2 what is your favorite book	my favorite is harry potter
"""

temp_file.write(sample_data)
temp_file.close()

ARGS = {
    'split_dir': temp_file_path
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type tuple, ndim 2, dtype family float — a tuple output has no array shape of its own; none is stated
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
set_tokenier_and_model Ran
ran on a synthesized input
songhaoyu/BoB/bertoverbert.py
code served (permissive licence) · get_code("2d2b420c01bf3bdc")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned tuple, ndim 1, dtype family float; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn

torch.manual_seed(0)

class MockTokenizer:
    def __init__(self):
        self.bos_token = None
        self.eos_token = None
        self.cls_token = "[CLS]"
        self.sep_token = "[SEP]"
        self.pad_token_id = 0
        self.bos_token_id = 101
        self.eos_token_id = 102

class MockDecoderConfig:
    def __init__(self):
        self.vocab_size = 30522

class MockConfig:
    def __init__(self):
        self.decoder_start_token_id = None
        self.eos_token_id = None
        self.pad_token_id = None
        self.vocab_size = None
        self.max_length = None
        self.min_length = None
        self.no_repeat_ngram_size = None
        self.early_stopping = None
        self.length_penalty = None
        self.num_beams = None
        self.temperature = None
        self.output_hidden_states = None
        self.decoder = MockDecoderConfig()

class MockModel(nn.Module):
    def __init__(self):
        super().__init__()
        self.config = MockConfig()
        self.linear = nn.Linear(4, 4)
    
    def forward(self, x):
        return self.linear(x)

tokenizer = MockTokenizer()
model = MockModel()

ARGS = {
    "tokenizer": tokenizer,
    "model": model
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type tuple, ndim 1, dtype family float — a tuple output has no array shape of its own; none is stated
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
shift_tokens_right Ran
ran; our contract draft was wrong, not the code
songhaoyu/BoB/xlibs/modeling_bart.py
pointer only (licence: Apache-2.0); the stored copy get_code serves, from yeliu918/KG-BART, is pointer only (licence: NONE) · get_code("e00ccf7847d12b59")
How it ran

The body of this sample is withheld: this paper's link allows the body inline (Apache-2.0), but get_code serves the stored copy, first harvested from yeliu918/KG-BART, and GitHub detected no licence in that repository when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

corpus_bleu Not yet run
unverified (harvested, no recorded run)
songhaoyu/BoB/evaluations.py
code served (permissive licence) · get_code("f6c5033f9640b16f")
How it ran

no invocation recorded.

modified_precision Not yet run
unverified (harvested, no recorded run)
songhaoyu/BoB/evaluations.py
code served (permissive licence) · get_code("d52e5869b9bdda18")
How it ran

no invocation recorded.

read_ecdt2019_split Not yet run
unverified (harvested, no recorded run)
songhaoyu/BoB/dataloader.py
code served (permissive licence) · get_code("21bf793899236407")
How it ran

no invocation recorded.

read_nli_split Not yet run
unverified (harvested, no recorded run)
songhaoyu/BoB/dataloader.py
code served (permissive licence) · get_code("dd28ddd90347931e")
How it ran

no invocation recorded.

Also present in other papers' code

Of 12 samples lifted from this paper's linked repositories, 12 carry a function identity and 0 do not. Of the 12 distinct functions with an identity, 6 are also present in other papers whose code we hold (6 with at least one paper dated earlier, 0 the same month, 4 later) and 6 are only here among the 52,977 papers whose code Syntology holds (as of 2026-09-26).

Only here means: among the 52,977 papers whose code Syntology holds (51,887 with lifted samples, 1,090 through their assemblies only), as of 2026-09-26, no other paper's lifted samples or current assembly bricks contain an identical function body after normalisation.

FunctionAlso present inEarliest-datedOthers named
shift_tokens_right 11 other papers
4 dated earlier, 5 later, 2 same year, order not determinable
1910.13461 (2019-10)
BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension
2009.12005 (2020-09)
MinTL: Minimalist Transfer Learning for Task-Oriented Dialogue Systems
2009.12677 (2020-09)
KG-BART: Knowledge Graph-Augmented BART for Generative Commonsense Reasoning
and 8 more not named here
invert_mask 10 other papers
4 dated earlier, 4 later, 2 same year, order not determinable
1910.13461 (2019-10)
BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension
2009.12005 (2020-09)
MinTL: Minimalist Transfer Learning for Task-Oriented Dialogue Systems
2009.12677 (2020-09)
KG-BART: Knowledge Graph-Augmented BART for Generative Commonsense Reasoning
and 7 more not named here
pad_sequence 10 other papers
3 dated earlier, 7 later
1909.09436 (2019-09)
CodeSearchNet Challenge: Evaluating the State of Semantic Code Search
1911.04700 (2019-11)
A Pre-training Based Personalized Dialogue Generation Model with Persona-sparse Data
2009.10297 (2020-09)
CodeBLEU: a Method for Automatic Evaluation of Code Synthesis
and 7 more not named here
make_padding_mask 9 other papers
4 dated earlier, 3 later, 2 same year, order not determinable
1910.13461 (2019-10)
BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension
2009.12005 (2020-09)
MinTL: Minimalist Transfer Learning for Task-Oriented Dialogue Systems
2009.12677 (2020-09)
KG-BART: Knowledge Graph-Augmented BART for Generative Commonsense Reasoning
and 6 more not named here
modified_precision 2 other papers
2 dated earlier
1911.04700 (2019-11)
A Pre-training Based Personalized Dialogue Generation Model with Persona-sparse Data
2009.07396 (2020-09)
Grounded Adaptation for Zero-shot Executable Semantic Parsing
corpus_bleu 1 other paper
1 dated earlier
2009.07396 (2020-09)
Grounded Adaptation for Zero-shot Executable Semantic Parsing

Presence, not direction: an identical function body in two papers' repositories does not say who wrote it, which came from which, or that either paper's method rests on it. Matching is exact after normalisation, so a renamed or edited function is not matched. Dates are when papers appeared, not when code was written. Only .py files the harvester lifted are covered; notebooks are not read. A new-style arXiv id YYMM.NNNN dates a paper to the month 20YY-MM; otherwise Paper.year dates it to the year; otherwise it is undated. Dates are when the paper appeared, not when the code was written. Read 2026-11-22.

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

Maintaining consistent personas is essential for dialogue agents. Although tremendous advancements have been brought, the limitedscale of annotated persona-dense data are still barriers towards training robust and consistent persona-based dialogue models. In this work, we show how the challenges can be addressed by disentangling persona-based dialogue generation into two sub-tasks with a novel BERTover-BERT (BoB) model. Specifically, the model consists of a BERT-based encoder and two BERT-based decoders, where one decoder is for response generation, and another is for consistency understanding. In particular, to learn the ability of consistency understanding from large-scale non-dialogue inference data, we train the second decoder in an unlikelihood manner. Under different limited data settings, both automatic and human evaluations demonstrate that the proposed model outperforms strong baselines in response quality and persona consistency.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2106.06169")
get_code_for_paper("2106.06169")
have("2106.06169")

The run record, dated, one paper per request, free:

curl https://syntology.ai/api/ran/2106.06169.json

A badge for a README (the split and the date, never a ratio):

[![Syntology run record](https://syntology.ai/api/ran/2106.06169.svg)](https://syntology.ai/paper/2106.06169)

Connect an agent — have() is free.