SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2110.06366 · 2021

Time Masking for Temporal Language Models

arXiv · PDF · Open in the Atlas

Code that ran

12 of 12 samples (functions and classes) lifted from this paper's linked repositories executed on a synthesized input in Syntology's jail (0 of them constructed an object rather than computing a result; 2 honoured the contract we drafted, 0 violated it and 8 have no contract verdict; 2 failures of our instrument; 0 no recorded run), measured 2026-11-21; repository links from the Papers with Code archive attributed under CC BY-SA, no affiliation; bodies inline only under a permissive licence as GitHub detects it. Not a reproduction of the paper's results.

"Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results. This record, dated, as JSON: /api/ran/2110.06366.json · badge: /api/ran/2110.06366.svg

Each repository's role is a label: a classifier's output recorded on the link, or, where no role is recorded, what the Papers with Code archive marked. The line under it names its basis. Neither says who wrote the code.

RepositoryRoleRan
guyrosin/tempobert canonical (classifier label)
classifier basis: classifier_2025
3 ran · 0 no recorded run
guyrosin/temporal_attention community (classifier label)
classifier basis not recorded on the link
9 ran · 0 no recorded run
FunctionStatusWhere it lives
calc_change_score_cosine_dist Ran
ran on a synthesized input
guyrosin/temporal_attention/semantic_change_detection.py
code served (permissive licence) · get_code("4675a84ab7a3e543")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned float, ndim 0, dtype family float, shape () as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn

torch.manual_seed(0)

class MockModel(nn.Module):
    def __init__(self):
        super().__init__()
        self.embedding_dim = 4
    
    def embed_word(self, sentences, word, batch_size=None):
        """
        Returns embeddings for the word across all sentences.
        Mocking this to return a tensor of shape (num_sentences, embedding_dim)
        """
        # Return 2 embeddings of dimension 4
        return torch.randn(2, self.embedding_dim)
    
    def __repr__(self):
        return "MockModel"

# Create model instance
model = MockModel()

# Create sentences (list of strings)
sentences = ["the word appears here", "word shows up again"]

# Word to analyze
word = "word"

# Optional batch size
batch_size = None

# Verbose flag
verbose = False

ARGS = {
    "model": model,
    "sentences": sentences,
    "word": word,
    "batch_size": batch_size,
    "verbose": verbose,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
loguru, torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type float, ndim 0, dtype family float, shape () — shape () as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
config_to_temporal Ran
ran on a synthesized input
guyrosin/temporal_attention/hf_utils.py
code served (permissive licence) · get_code("441f6740f7809dfb")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned TemporalConfig. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn

torch.manual_seed(0)

# Create a minimal config object
class MinimalConfig:
    pass

config = MinimalConfig()

# Define the class to assign to config.__class__
class TemporalConfig:
    pass

# Define times - can be a tensor, list, or array
times = torch.tensor([0.0, 1.0, 2.0, 3.0])

# Define time_embedding_type - a string specifying the type
time_embedding_type = "sinusoidal"

ARGS = {
    "config": config,
    "cls": TemporalConfig,
    "times": times,
    "time_embedding_type": time_embedding_type
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type TemporalConfig — a TemporalConfig output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a TemporalConfig output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
get_embedding Ran
ran, honoured the contract we drafted
guyrosin/tempobert/semantic_change_detection.py
code served (permissive licence) · get_code("f5f8f6ed8c77fc76")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1) and returned Tensor, ndim 1, dtype family float, shape (4,) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn

torch.manual_seed(0)

# Stand-in tokenizer with vocab
class StandInTokenizer:
    def __init__(self):
        self.vocab = {"hello": 1, "world": 2, "test": 3}

# Stand-in config
class StandInConfig:
    def __init__(self, num_hidden_layers=12):
        self.num_hidden_layers = num_hidden_layers

# Stand-in model with embed_word method
class StandInModel(nn.Module):
    def __init__(self):
        super().__init__()
        self.tokenizer = StandInTokenizer()
        self.config = StandInConfig(num_hidden_layers=12)
        self.embedding_layer = nn.Embedding(10, 4)
    
    def embed_word(self, sentences_loop, word, time=None, batch_size=None, hidden_layers_number=None):
        # Return a tensor of embeddings
        # Return shape (2, 4) to test the centroid calculation
        return torch.randn(2, 4)

model = StandInModel()

# sentences_loop: list of sentences (can be strings or tokenized)
sentences_loop = ["hello world", "test sentence"]

# word: a word to get embedding for
word = "hello"

# time: optional time parameter
time = None

# batch_size: optional batch size
batch_size = None

# require_word_in_vocab: whether to require word in vocab
require_word_in_vocab = False

# hidden_layers_number: optional number of hidden layers
hidden_layers_number = None

ARGS = {
    "model": model,
    "sentences_loop": sentences_loop,
    "word": word,
    "time": time,
    "batch_size": batch_size,
    "require_word_in_vocab": require_word_in_vocab,
    "hidden_layers_number": hidden_layers_number,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Tensor, ndim 1, dtype family float, shape (4,) — shape (4,) as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
get_embedding Ran
ran on a synthesized input
guyrosin/temporal_attention/semantic_change_detection.py
code served (permissive licence) · get_code("ccd9731bc1c63e5c")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned Tensor, ndim 1, dtype family float, shape (4,) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn

torch.manual_seed(0)

# Stand-in tokenizer with vocab
class StandInTokenizer:
    def __init__(self):
        self.vocab = {"hello": 1, "world": 2, "test": 3, "word": 4}

# Stand-in config
class StandInConfig:
    def __init__(self, num_hidden_layers=12):
        self.num_hidden_layers = num_hidden_layers

# Stand-in model
class StandInModel(nn.Module):
    def __init__(self):
        super().__init__()
        self.tokenizer = StandInTokenizer()
        self.config = StandInConfig(num_hidden_layers=12)
        self.embedding_layer = nn.Embedding(5, 4)
    
    def embed_word(self, sentences, word, time=None, batch_size=None, hidden_layers_number=None):
        # Return a tensor with shape (2, 4) for multiple embeddings or (4,) for single
        # Simulate returning embeddings for the word across sentences
        embs = torch.randn(2, 4)
        return embs

model = StandInModel()

# Test sentences
sentences = ["hello world", "test word"]

# Word to embed
word = "word"

# Optional parameters
time = None
batch_size = 2
require_word_in_vocab = False
hidden_layers_number = None

ARGS = {
    "model": model,
    "sentences": sentences,
    "word": word,
    "time": time,
    "batch_size": batch_size,
    "require_word_in_vocab": require_word_in_vocab,
    "hidden_layers_number": hidden_layers_number,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Tensor, ndim 1, dtype family float, shape (4,) — shape (4,) as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
lazyconfig_getitem Ran
ran on a synthesized input
guyrosin/temporal_attention/hf_utils.py
code served (permissive licence) · get_code("182620e7a1d9acc4")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned type. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import importlib
from enum import Enum, auto
from unittest.mock import MagicMock

class TempoModelName(Enum):
    TempoBert = auto()

    @classmethod
    def has_key(cls, name):
        return any((x for x in cls if x.name == name or x.name.lower() == name))

def lazyconfig_getitem(mapping, key):
    """
    Adapted from _LazyConfigMapping.__getitem__() in configuration_auto.py
    """
    if key not in mapping._mapping:
        raise KeyError(key)
    value = mapping._mapping[key]
    
    # Mock implementation of model_type_to_module_name
    def model_type_to_module_name(key):
        return key.replace("-", "_")
    
    module_name = model_type_to_module_name(key)
    if module_name not in mapping._modules:
        if TempoModelName.has_key(key):
            mapping._modules[module_name] = importlib.import_module(f'.{module_name}', 'models')
        else:
            mapping._modules[module_name] = importlib.import_module(f'.{module_name}', 'transformers.models')
    return getattr(mapping._modules[module_name], value)

# Create a mock mapping object
mock_mapping = MagicMock()
mock_mapping._mapping = {
    'bert': 'BertConfig',
    'tempobert': 'TempoBertConfig'
}
mock_mapping._modules = {}

# For standard transformers models, we need to mock the module
mock_bert_module = MagicMock()
mock_bert_module.BertConfig = type('BertConfig', (), {})

# Inject the mock module
mock_mapping._modules['bert'] = mock_bert_module

ARGS = {
    'mapping': mock_mapping,
    'key': 'bert'
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch, transformers — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type type — a type output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a type output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
millify Ran
ran on a synthesized input
guyrosin/temporal_attention/utils.py
code served (permissive licence) · get_code("4948b19066b3d98c")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned str. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import math

def millify(num, precision=0):
    """Humanize number (e.g., 30000 --> 30k).
    Taken from https://github.com/azaitsev/millify/tree/master/millify"""
    millnames = ['', 'k', 'M', 'B', 'T', 'P', 'E', 'Z', 'Y']
    num = float(num)
    millidx = max(0, min(len(millnames) - 1, int(math.floor(0 if num == 0 else math.log10(abs(num)) / 3))))
    result = '{:.{precision}f}'.format(num / 10 ** (3 * millidx), precision=precision)
    return '{0}{dx}'.format(result, dx=millnames[millidx])


ARGS = {
    'num': 1500000,
    'precision': 2
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type str — a str output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a str output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
noop Ran
ran, honoured the contract we drafted
guyrosin/temporal_attention/tokenization_utils_fast.py
code served (permissive licence) · get_code("7dc7a4c0ec126f86")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned Tensor, ndim 2, dtype family float, shape (2, 4) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn

torch.manual_seed(0)

# The function noop has parameters: x (default None), *args, **kwargs
# The function body simply returns x, so it doesn't actually use any parameters
# We just need to provide valid values for these parameters

ARGS = {
    "x": torch.randn(2, 4),
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Tensor, ndim 2, dtype family float, shape (2, 4) — shape (2, 4) as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
prepend Ran
ran on a synthesized input
guyrosin/temporal_attention/tokenization_utils_fast.py
code served (permissive licence) · get_code("6bcecd4cd8678430")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned list, ndim 1, dtype family float; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch

def noop(x=None, *args, **kwargs):
    """Do nothing"""
    return x

def prepend(texts, time_ids, stringify_time_id=noop):
    """Prepend the time to each text"""
    texts = [f'{stringify_time_id(time_id)} {text}' for text, time_id in zip(texts, time_ids)]
    return texts

torch.manual_seed(0)

# texts: list of strings
texts = ["hello world", "foo bar"]

# time_ids: list of values that will be passed to stringify_time_id
time_ids = [1, 2]

# stringify_time_id: a callable that takes a time_id and returns a string representation
# Using the default noop function as provided
stringify_time_id = noop

ARGS = {
    "texts": texts,
    "time_ids": time_ids,
    "stringify_time_id": stringify_time_id,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type list, ndim 1, dtype family float — a list output has no array shape of its own; none is stated
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
prepend_tuples Ran
ran on a synthesized input
guyrosin/temporal_attention/tokenization_utils_fast.py
code served (permissive licence) · get_code("ffe2a285827399df")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned list, ndim 2, dtype family float; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import numpy as np

def noop(x=None, *args, **kwargs):
    """Do nothing"""
    return x

def prepend(texts, time_ids, stringify_time_id=noop):
    """Prepend the time to each text"""
    texts = [f'{stringify_time_id(time_id)} {text}' for text, time_id in zip(texts, time_ids)]
    return texts

def prepend_tuples(text_pairs, time_id_pairs, stringify_time_id=noop):
    """Prepend the time to each text, where the inputs are pairs"""
    texts = [prepend(text_pair, time_id_pair, stringify_time_id) for text_pair, time_id_pair in zip(text_pairs, time_id_pairs)]
    return texts

torch.manual_seed(0)

# Define sample data
text_pairs = [
    ["hello world", "goodbye world"],
    ["foo bar", "baz qux"]
]

time_id_pairs = [
    [1, 2],
    [3, 4]
]

def custom_stringify(time_id):
    """Custom function to stringify time_id"""
    return f"[{time_id}]"

ARGS = {
    "text_pairs": text_pairs,
    "time_id_pairs": time_id_pairs,
    "stringify_time_id": custom_stringify
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type list, ndim 2, dtype family float — a list output has no array shape of its own; none is stated
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
search_sequence_numpy Ran
ran on a synthesized input
guyrosin/temporal_attention/utils.py
code served (permissive licence) · get_code("7bbcb9444f86f876")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned ndarray, ndim 1, dtype family int, shape (9,) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import numpy as np

# Create test data for search_sequence_numpy
# arr: a 1D array to search in
# seq: a 1D sequence to find

arr = np.array([1, 2, 3, 4, 5, 3, 4, 5, 6, 7, 3, 4, 5])
seq = np.array([3, 4, 5])

ARGS = {
    "arr": arr,
    "seq": seq,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
numpy — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type ndarray, ndim 1, dtype family int, shape (9,) — shape (9,) as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
tokenize_dataset_concat Ran
ran; our contract draft was wrong, not the code
guyrosin/tempobert/train_tempobert.py
code served (permissive licence) · get_code("ac64b4860d21447a")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1) and returned MockDataset. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import numpy as np
from unittest.mock import Mock, MagicMock
from contextlib import contextmanager

torch.manual_seed(0)

# Create a minimal tokenizer stand-in
class MockTokenizer:
    def __call__(self, texts, times, return_special_tokens_mask=False):
        # Return a dict with input_ids, attention_mask, and special_tokens_mask
        batch_size = len(texts) if isinstance(texts, list) else 1
        seq_len = 10
        return {
            'input_ids': [[101] * seq_len for _ in range(batch_size)],
            'attention_mask': [[1] * seq_len for _ in range(batch_size)],
            'special_tokens_mask': [[0] * seq_len for _ in range(batch_size)] if return_special_tokens_mask else None,
        }

# Create a minimal dataset stand-in
class MockDataset:
    def __init__(self):
        self.data = {
            'text': ['hello world', 'foo bar'],
            'time': ['2021-01-01', '2021-01-02'],
        }
    
    def map(self, func, batched=False, num_proc=None, remove_columns=None, load_from_cache_file=False, desc=None):
        # Simulate dataset.map by applying function and returning a new dataset
        result = MockDataset()
        if batched:
            batch = {k: v for k, v in self.data.items()}
            processed = func(batch)
            result.data = processed
        else:
            result.data = self.data
        return result

# Create minimal training_args with main_process_first context manager
class MockTrainingArgs:
    def __init__(self):
        self.preprocessing_num_workers = 1
        self.overwrite_cache = False
    
    @contextmanager
    def main_process_first(self, desc=None):
        yield

# Create minimal data_args
class MockDataArgs:
    def __init__(self):
        self.preprocessing_num_workers = 1
        self.overwrite_cache = False

# Build the arguments dictionary
ARGS = {
    'dataset': MockDataset(),
    'data_args': MockDataArgs(),
    'training_args': MockTrainingArgs(),
    'tokenizer': MockTokenizer(),
    'text_column_name': 'text',
    'column_names': ['text', 'time'],
    'max_seq_length': 128,
    'return_special_tokens_mask': True,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1
Imports
numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type MockDataset — a MockDataset output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a MockDataset output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
tokenize_dataset_line_by_line Ran
ran; our contract draft was wrong, not the code
guyrosin/tempobert/train_tempobert.py
code served (permissive licence) · get_code("e3702536b9a34fea")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned MockDataset. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import numpy as np
from torch import nn
from unittest.mock import MagicMock

torch.manual_seed(0)

# Mock tokenizer
class MockTokenizer:
    def __call__(self, texts, times, padding=False, truncation=False, max_length=None, return_special_tokens_mask=False):
        batch_size = len(texts)
        return {
            'input_ids': torch.randint(0, 1000, (batch_size, 10)).tolist(),
            'attention_mask': torch.ones(batch_size, 10).tolist(),
            'special_tokens_mask': torch.zeros(batch_size, 10).tolist() if return_special_tokens_mask else None,
        }

# Mock data_args
class MockDataArgs:
    def __init__(self):
        self.pad_to_max_length = False
        self.preprocessing_num_workers = 1
        self.overwrite_cache = False

# Mock training_args
class MockTrainingArgs:
    def main_process_first(self, desc=None):
        class ContextManager:
            def __enter__(self):
                return self
            def __exit__(self, *args):
                pass
        return ContextManager()

# Mock dataset
class MockDataset:
    def map(self, function, batched=False, num_proc=1, remove_columns=None, load_from_cache_file=True, desc=None):
        # Simulate mapping over a simple dataset
        sample_batch = {
            'text': ['hello world', 'test line', '   '],
            'time': [1, 2, 3],
        }
        function(sample_batch)
        return self

# Construct arguments
ARGS = {
    'dataset': MockDataset(),
    'data_args': MockDataArgs(),
    'training_args': MockTrainingArgs(),
    'tokenizer': MockTokenizer(),
    'text_column_name': 'text',
    'column_names': ['text', 'time'],
    'max_seq_length': 128,
    'return_special_tokens_mask': True,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type MockDataset — a MockDataset output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a MockDataset output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block

Also present in other papers' code

Of 12 samples lifted from this paper's linked repositories, 12 carry a function identity and 0 do not. Of the 12 distinct functions with an identity, 1 are also present in other papers whose code we hold (0 with at least one paper dated earlier, 0 the same month, 1 later) and 11 are only here among the 52,977 papers whose code Syntology holds (as of 2026-09-26).

Only here means: among the 52,977 papers whose code Syntology holds (51,887 with lifted samples, 1,090 through their assemblies only), as of 2026-09-26, no other paper's lifted samples or current assembly bricks contain an identical function body after normalisation.

FunctionAlso present inEarliest-datedOthers named
noop 1 other paper
1 later
2212.09840 (2022-12)
Dynamic Sparse Network for Time Series Classification: Learning What to "See"

Presence, not direction: an identical function body in two papers' repositories does not say who wrote it, which came from which, or that either paper's method rests on it. Matching is exact after normalisation, so a renamed or edited function is not matched. Dates are when papers appeared, not when code was written. Only .py files the harvester lifted are covered; notebooks are not read. A new-style arXiv id YYMM.NNNN dates a paper to the month 20YY-MM; otherwise Paper.year dates it to the year; otherwise it is undated. Dates are when the paper appeared, not when the code was written. Read 2026-11-21.

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

Our world is constantly evolving, and so is the content on the web. Consequently, our languages, often said to mirror the world, are dynamic in nature. However, most current contextual language models are static and cannot adapt to changes over time. In this work, we propose a temporal contextual language model called TempoBERT, which uses time as an additional context of texts. Our technique is based on modifying texts with temporal information and performing time masking - specific masking for the supplementary time information. We leverage our approach for the tasks of semantic change detection and sentence time prediction, experimenting on diverse datasets in terms of time, size, genre, and language. Our extensive evaluation shows that both tasks benefit from exploiting time masking.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2110.06366")
get_code_for_paper("2110.06366")
have("2110.06366")

The run record, dated, one paper per request, free:

curl https://syntology.ai/api/ran/2110.06366.json

A badge for a README (the split and the date, never a ratio):

[![Syntology run record](https://syntology.ai/api/ran/2110.06366.svg)](https://syntology.ai/paper/2110.06366)

Connect an agent — have() is free.