SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2210.02414 · ICLR · 2023

GLM-130B: An Open Bilingual Pre-trained Model

Yuxiao Dong, Jie Tang, Ming Ding, Zhuoyi Yang, Wendi Zheng, Xiao Liu, Zihan Wang, Aohan Zeng, Zhengxiao Du, Yifan Xu, Xiao Xia, Weng Lam, and 5 more

arXiv · PDF · Open in the Atlas

Code that ran

15 of 21 samples (functions and classes) lifted from this paper's linked repositories executed on a synthesized input in Syntology's jail (0 of them constructed an object rather than computing a result; 0 honoured the contract we drafted, 0 violated it and 11 have no contract verdict; 4 failures of our instrument; 6 no recorded run), measured 2026-11-22; repository links from the Papers with Code archive attributed under CC BY-SA, no affiliation; bodies inline only under a permissive licence as GitHub detects it. Not a reproduction of the paper's results.

"Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results. This record, dated, as JSON: /api/ran/2210.02414.json · badge: /api/ran/2210.02414.svg

Each repository's role is a label: a classifier's output recorded on the link, or, where no role is recorded, what the Papers with Code archive marked. The line under it names its basis. Neither says who wrote the code.

RepositoryRoleRan
THUDM/GLM-130B canonical (classifier label)
classifier basis not recorded on the link
9 ran · 0 no recorded run
jackaduma/ChatGLM-LoRA-RLHF-PyTorch community (classifier label)
classifier basis not recorded on the link
4 ran · 2 no recorded run
THUDM/GLM community
Papers with Code archive
2 ran · 2 no recorded run
thudm/chatglm3 community
Papers with Code archive
0 ran · 2 no recorded run
FunctionStatusWhere it lives
F1_metric Ran
ran on a synthesized input
THUDM/GLM-130B/evaluation/metrics.py
code served (permissive licence) · get_code("57bdd79eeeb0d09e")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned float, ndim 0, dtype family float, shape () as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import re
import string
from collections import Counter

def normalize_answer(s):
    """Lower text and remove punctuation, articles and extra whitespace."""

    def remove_articles(text):
        return re.sub('\\b(a|an|the)\\b', ' ', text)

    def white_space_fix(text):
        return ' '.join(text.split())

    def remove_punc(text):
        exclude = set(string.punctuation)
        return ''.join((ch for ch in text if ch not in exclude))

    def lower(text):
        return text.lower()
    return white_space_fix(remove_articles(remove_punc(lower(s))))

def f1_score(prediction, ground_truth):
    prediction_tokens = normalize_answer(prediction).split()
    ground_truth_tokens = normalize_answer(ground_truth).split()
    common = Counter(prediction_tokens) & Counter(ground_truth_tokens)
    num_same = sum(common.values())
    if num_same == 0:
        return 0
    precision = 1.0 * num_same / len(prediction_tokens)
    recall = 1.0 * num_same / len(ground_truth_tokens)
    f1 = 2 * precision * recall / (precision + recall)
    return f1

def F1_metric(predictions, examples):
    assert len(predictions) == len(examples)
    from sklearn.metrics import f1_score
    truth = []
    for prediction, example in zip(predictions, examples):
        truth.append(example['label'])
    return f1_score(truth, predictions, average='micro') * 100.0

# Setup code to construct valid arguments
predictions = [0, 1, 0, 1]
examples = [
    {'label': 0},
    {'label': 1},
    {'label': 0},
    {'label': 1}
]

ARGS = {
    'predictions': predictions,
    'examples': examples
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
sklearn — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type float, ndim 0, dtype family float, shape () — shape () as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
PromptSpell Ran
ran on a synthesized input
THUDM/GLM/model/modeling_glm.py
code served (permissive licence) · get_code("184ed80cb890ac12")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, class construct-then-call prompt, model not recorded) and returned Tensor, ndim 3, dtype family float; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import random

class PromptSpell(torch.nn.Module):

    def __init__(self, spell_length, hidden_size, spell_func):
        super(PromptSpell, self).__init__()
        self.spell_length = spell_length
        self.hidden_size = hidden_size
        self.spell_embeddings = torch.nn.Embedding(self.spell_length, self.hidden_size)
        self.spell_func = spell_func
        if self.spell_func == 'lstm':
            self.lstm_head = torch.nn.LSTM(input_size=self.hidden_size, hidden_size=self.hidden_size, num_layers=2, bidirectional=True, batch_first=True)
            self.mlp_head = torch.nn.Sequential(torch.nn.Linear(2 * self.hidden_size, self.hidden_size), torch.nn.ReLU(), torch.nn.Linear(self.hidden_size, self.hidden_size))
        elif self.spell_func == 'mlp':
            self.mlp_head = torch.nn.Sequential(torch.nn.Linear(self.hidden_size, self.hidden_size), torch.nn.ReLU(), torch.nn.Linear(self.hidden_size, self.hidden_size))
        elif self.spell_func != 'none':
            raise NotImplementedError('Prompt function ' + self.spell_func)

    def init_embedding(self, word_embeddings=None, task_tokens=None):
        num_words = 5000
        with torch.no_grad():
            for i in range(self.spell_length):
                rand_token = random.randrange(num_words)
                if task_tokens is None:
                    target_embedding = word_embeddings[rand_token]
                else:
                    word_embedding = word_embeddings[rand_token]
                    task_token = random.choice(task_tokens)
                    task_embedding = word_embeddings[task_token]
                    ratio = random.random()
                    target_embedding = word_embedding * ratio + task_embedding * (1 - ratio)
                self.spell_embeddings.weight.data[i] = target_embedding

    def forward(self):
        prompt_embeds = self.spell_embeddings.weight.unsqueeze(0)
        if self.spell_func == 'lstm':
            prompt_embeds = self.lstm_head(prompt_embeds)[0]
        if self.spell_func == 'lstm' or self.spell_func == 'mlp':
            prompt_embeds = self.mlp_head(prompt_embeds)
        return prompt_embeds


INIT_ARGS = {
    'spell_length': 8,
    'hidden_size': 4,
    'spell_func': 'mlp'
}

ARGS = {}

if __name__ == '__main__':
    model = PromptSpell(**INIT_ARGS)
    output = model(**ARGS)
    print("Output shape:", output.shape)
    print("Output:", output)
Setup block drafted by
a language model, class construct-then-call prompt, model not recorded
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Tensor, ndim 3, dtype family float — no output shape is stated: observe_signature.py could not observe it (no_inputs)
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: torch and numpy seeded to 0 by the harness immediately before the class was constructed from INIT_ARGS (fixture_check.py)
accuracy_metric Ran
ran on a synthesized input
THUDM/GLM-130B/evaluation/metrics.py
code served (permissive licence) · get_code("1bc1a496ad4d784f")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned float, ndim 0, dtype family float, shape () as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch

torch.manual_seed(0)

# predictions: list of predicted labels
# examples: list of dicts with 'label' key

predictions = [1, 0, 1, 0]
examples = [
    {'label': 1},
    {'label': 0},
    {'label': 1},
    {'label': 1}
]

ARGS = {
    'predictions': predictions,
    'examples': examples
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type float, ndim 0, dtype family float, shape () — shape () as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
add_bminf_args Ran
ran; our contract draft was wrong, not the code
THUDM/GLM-130B/initialize.py
code served (permissive licence) · get_code("945d1222f0c75dab")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned ArgumentParser. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import argparse
import torch
import torch.nn as nn

torch.manual_seed(0)

# Create a minimal parser that can accept argument groups
parser = argparse.ArgumentParser()

# ARGS dictionary mapping the parameter 'parser' to an ArgumentParser instance
ARGS = {
    'parser': parser
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type ArgumentParser — a ArgumentParser output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a ArgumentParser output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
add_evaluation_specific_args Ran
ran on a synthesized input
THUDM/GLM-130B/evaluate.py
code served (permissive licence) · get_code("8c1dee88ecad5be5")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned ArgumentParser. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import argparse
import tempfile
import os

# Create a minimal parser
parser = argparse.ArgumentParser()

# The function add_evaluation_specific_args expects a parser object
# and adds argument group to it. We need to pass a valid ArgumentParser instance.

# Create a temporary directory for the data-path
temp_dir = tempfile.mkdtemp()

ARGS = {
    'parser': parser
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type ArgumentParser — a ArgumentParser output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a ArgumentParser output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
find_all_tasks Ran
ran on a synthesized input
THUDM/GLM-130B/evaluate.py
code served (permissive licence) · get_code("1a1a6bb1bc387926")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned list, ndim 1, dtype family float; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import os
import tempfile
import shutil

# Create a temporary directory structure with some yaml files for testing
temp_dir = tempfile.mkdtemp()
task_dir1 = os.path.join(temp_dir, 'tasks1')
task_dir2 = os.path.join(temp_dir, 'tasks2')
os.makedirs(os.path.join(task_dir1, 'subdir'), exist_ok=True)
os.makedirs(task_dir2, exist_ok=True)

# Create some dummy yaml files
with open(os.path.join(task_dir1, 'task1.yaml'), 'w') as f:
    f.write('task: 1')
with open(os.path.join(task_dir1, 'subdir', 'task2.yaml'), 'w') as f:
    f.write('task: 2')
with open(os.path.join(task_dir2, 'task3.yaml'), 'w') as f:
    f.write('task: 3')

# Create a single file path
single_file = os.path.join(temp_dir, 'single_task.yaml')
with open(single_file, 'w') as f:
    f.write('task: single')

ARGS = {
    'all_task_config_path': [task_dir1, task_dir2, single_file]
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type list, ndim 1, dtype family float — a list output has no array shape of its own; none is stated
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
format_example Ran
ran on a synthesized input
jackaduma/ChatGLM-LoRA-RLHF-PyTorch/cover_alpaca2jsonl.py
code served (permissive licence) · get_code("1b801af1b3425cfa")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned dict. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch

ARGS = {
    'example': {
        'instruction': 'What is the capital of France?',
        'input': 'France is a country in Europe.',
        'output': 'The capital of France is Paris.'
    }
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type dict — a dict output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a dict output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
gelu Ran
ran; the contract check could not run (RAISES, DEP_MISSING or TIMEOUT)
jackaduma/ChatGLM-LoRA-RLHF-PyTorch/models/modeling_chatglm.py
code served (permissive licence) · get_code("43c9688838aff8b2")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned Tensor, ndim 2, dtype family float; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn

torch.manual_seed(0)

# The function gelu takes a single parameter: x
# Looking at the body, x is used in arithmetic operations (multiplication, addition, tanh)
# so it needs to be a tensor

# Create a small tensor for testing
x = torch.randn(2, 4)

ARGS = {
    "x": x
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Tensor, ndim 2, dtype family float — no output shape is stated: observe_signature.py could not observe it (import:OSError)
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
gelu_impl Ran
ran; the contract check could not run (RAISES, DEP_MISSING or TIMEOUT)
jackaduma/ChatGLM-LoRA-RLHF-PyTorch/models/modeling_chatglm.py
code served (permissive licence) · get_code("c8f235d13277facd")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned Tensor, ndim 2, dtype family float; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch

torch.manual_seed(0)

def gelu_impl(x):
    """OpenAI's gelu implementation."""
    return 0.5 * x * (1.0 + torch.tanh(0.7978845608028654 * x * (1.0 + 0.044715 * x * x)))

# Create a small tensor for input
x = torch.randn(2, 4)

ARGS = {
    "x": x
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Tensor, ndim 2, dtype family float — no output shape is stated: observe_signature.py could not observe it (import:OSError)
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
get_masks_and_position_ids Ran
ran on a synthesized input
THUDM/GLM-130B/generate.py
code served (permissive licence) · get_code("5530d6c17cee956e")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned tuple. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch

torch.manual_seed(0)

# Construct a small sequence tensor
# seq is used for: seq.shape[1] to get context_length, and passed to pad
# We need batch_size=1 (implied by attention_mask construction), seq_length=4
seq = torch.randint(0, 100, (1, 4), dtype=torch.long)

# mask_position is used as a scalar value to assign to position_ids
# It's an integer position index
mask_position = 2

# max_gen_length is used in padding operation
# It determines how many tokens to pad
max_gen_length = 5

# gmask is a boolean flag that controls whether to set position_ids
gmask = False

ARGS = {
    "seq": seq,
    "mask_position": mask_position,
    "max_gen_length": max_gen_length,
    "gmask": gmask,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type tuple — a tuple output has no array shape of its own; none is stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
init_method_normal Ran
ran; our contract draft was wrong, not the code
THUDM/GLM/model/modeling_glm.py
code served (permissive licence) · get_code("6bcca529f47d7b8e")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1) and returned function. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn

torch.manual_seed(0)

ARGS = {
    "std": 0.02,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type function — a function output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a function output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
isEnglish Ran
ran on a synthesized input
THUDM/GLM-130B/generate.py
code served (permissive licence) · get_code("294d09efc94ec53e")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned bool, ndim 0, dtype family float, shape () as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import numpy as np

def isEnglish(s):
    try:
        s.encode(encoding='utf-8').decode('ascii')
    except UnicodeDecodeError:
        return False
    else:
        return True

# The function takes a single parameter 's' which should be a string
# It attempts to encode it as UTF-8 and decode as ASCII
# Valid English strings will succeed, non-ASCII strings will raise UnicodeDecodeError

ARGS = {
    's': 'Hello World'
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type bool, ndim 0, dtype family float, shape () — shape () as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
pad_batch Ran
ran on a synthesized input
THUDM/GLM-130B/evaluation/dataset.py
code served (permissive licence) · get_code("4cd519b7c609ddfa")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned tuple, ndim 2, dtype family int; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import numpy as np

# Analyze the function signature and body
# pad_batch takes: tokens, position_ids, attention_mask, max_seq_length
# - tokens: array that gets concatenated with zeros, has len() called on it
# - position_ids: array that gets concatenated with zeros, has len() called on it
# - attention_mask: array that gets padded, has len() called on it
# - max_seq_length: scalar int used as padding target

# Create small test arrays
current_seq_length = 3
max_seq_length = 5

tokens = np.array([1, 2, 3], dtype=np.int64)
position_ids = np.array([0, 1, 2], dtype=np.int64)
attention_mask = np.array([1, 1, 1], dtype=np.int64)

ARGS = {
    "tokens": tokens,
    "position_ids": position_ids,
    "attention_mask": attention_mask,
    "max_seq_length": max_seq_length
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
numpy — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type tuple, ndim 2, dtype family int — a tuple output has no array shape of its own; none is stated
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
precision_metric Ran
ran on a synthesized input
THUDM/GLM-130B/evaluation/metrics.py
code served (permissive licence) · get_code("cef36ffea76c70c8")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned float, ndim 0, dtype family float, shape () as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import numpy as np
from sklearn.metrics import precision_score

torch.manual_seed(0)

# Construct predictions: a list of predicted labels (0 or 1 for binary classification)
predictions = [0, 1, 1, 0]

# Construct examples: a list of dicts, each containing a 'label' key
examples = [
    {'label': 0},
    {'label': 1},
    {'label': 1},
    {'label': 0}
]

ARGS = {
    'predictions': predictions,
    'examples': examples
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
numpy, sklearn, torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type float, ndim 0, dtype family float, shape () — shape () as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
preprocess Ran
ran on a synthesized input
jackaduma/ChatGLM-LoRA-RLHF-PyTorch/tokenize_dataset_rows.py
code served (permissive licence) · get_code("2df80eb832dfdf12")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned dict. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import numpy as np

torch.manual_seed(0)

class MockTokenizer:
    def encode(self, text, max_length=None, truncation=False, add_special_tokens=True):
        # Simple mock: return token IDs based on text length
        tokens = [101] + [1000 + i for i in range(len(text) // 5)] + [102]
        if max_length and len(tokens) > max_length:
            tokens = tokens[:max_length]
        if not add_special_tokens and tokens and tokens[-1] == 102:
            tokens = tokens[:-1]
        return tokens

class MockConfig:
    def __init__(self):
        self.eos_token_id = 103

tokenizer = MockTokenizer()
config = MockConfig()
example = {
    'context': 'This is a prompt',
    'target': 'This is a target'
}
max_seq_length = 128

ARGS = {
    'tokenizer': tokenizer,
    'config': config,
    'example': example,
    'max_seq_length': max_seq_length
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type dict — a dict output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a dict output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
GLMModel Not yet run
unverified (harvested, no recorded run)
THUDM/GLM/model/modeling_glm.py
code served (permissive licence) · get_code("c9043275a83fa6fd")
How it ran

no invocation recorded.

load_tf_weights_in_chatglm_6b Not yet run
unverified (harvested, no recorded run)
jackaduma/ChatGLM-LoRA-RLHF-PyTorch/models/modeling_chatglm.py
code served (permissive licence) · get_code("25fd6233234767aa")
How it ran

no invocation recorded.

load_tokenizer_and_model Not yet run
unverified (harvested, no recorded run)
thudm/chatglm3/finetune_demo/finetune_hf.py
code served (permissive licence) · get_code("632dd3e505d6e594")
How it ran

no invocation recorded.

preprocess_function Not yet run
unverified (harvested, no recorded run)
jackaduma/ChatGLM-LoRA-RLHF-PyTorch/train_reward_model.py
code served (permissive licence) · get_code("732abe979cb6bf39")
How it ran

no invocation recorded.

print_model_size Not yet run
unverified (harvested, no recorded run)
thudm/chatglm3/finetune_demo/finetune_hf.py
code served (permissive licence) · get_code("677f282a0ca855b5")
How it ran

no invocation recorded.

print_rank_0 Not yet run
unverified (harvested, no recorded run)
THUDM/GLM/model/modeling_glm.py
code served (permissive licence) · get_code("58b8c47b2ea3036c")
How it ran

no invocation recorded.

Also present in other papers' code

Of 21 samples lifted from this paper's linked repositories, 21 carry a function identity and 0 do not. Of the 21 distinct functions with an identity, 10 are also present in other papers whose code we hold (2 with at least one paper dated earlier, 0 the same month, 9 later) and 11 are only here among the 52,977 papers whose code Syntology holds (as of 2026-09-26).

Only here means: among the 52,977 papers whose code Syntology holds (51,887 with lifted samples, 1,090 through their assemblies only), as of 2026-09-26, no other paper's lifted samples or current assembly bricks contain an identical function body after normalisation.

FunctionAlso present inEarliest-datedOthers named
gelu_impl 4 other papers
4 later
2307.13528 (2023-07)
FacTool: Factuality Detection in Generative AI -- A Tool Augmented Framework for Multi-Task and Multi-Domain Scenarios
2310.18339 (2023-10)
When MOE Meets LLMs: Parameter Efficient Fine-tuning for Multi-task Medical Applications
2411.04168 (2024-11)
DiMSUM: Diffusion Mamba - A Scalable and Unified Spatial-Frequency Method for Image Generation
and 1 more not named here
gelu 3 other papers
3 later
2307.13528 (2023-07)
FacTool: Factuality Detection in Generative AI -- A Tool Augmented Framework for Multi-Task and Multi-Domain Scenarios
2310.18339 (2023-10)
When MOE Meets LLMs: Parameter Efficient Fine-tuning for Multi-task Medical Applications
2025.findings-acl.583 (2025 (year only))
CitaLaw: Enhancing LLM with Citations in Legal Domain
load_tf_weights_in_chatglm_6b 3 other papers
3 later
2307.13528 (2023-07)
FacTool: Factuality Detection in Generative AI -- A Tool Augmented Framework for Multi-Task and Multi-Domain Scenarios
2310.18339 (2023-10)
When MOE Meets LLMs: Parameter Efficient Fine-tuning for Multi-task Medical Applications
2025.findings-acl.583 (2025 (year only))
CitaLaw: Enhancing LLM with Citations in Legal Domain
load_tokenizer_and_model
brick of this paper's assembly: entry
3 other papers
3 later
2406.05925 (2024-06)
Hello Again! LLM-powered Personalized Agent for Long-term Dialogue
2406.12793 (2024-06)
ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Tools
2025.naacl-long.272 (2025 (year only))
Hello Again! LLM-powered Personalized Agent for Long-term Dialogue
print_rank_0
brick of this paper's assembly: helper
2 other papers
1 dated earlier, 1 later
2204.07496 (2022-04)
Improving Passage Retrieval with Zero-Shot Question Generation
2409.17481 (2024-09)
MaskLLM: Learnable Semi-Structured Sparsity for Large Language Models
PromptSpell
brick of this paper's assembly: helper
1 other paper
1 dated earlier · 1 only as an assembly brick
2103.10385 (2021-03)
GPT Understands, Too
add_bminf_args 1 other paper
1 later
2308.10252 (2023-08)
LMTuner: An user-friendly and highly-integrable Training Framework for fine-tuning Large Language Models
format_example 1 other paper
1 later
2306.06031 (2023-06)
FinGPT: Open-Source Financial Large Language Models
init_method_normal
brick of this paper's assembly: helper
1 other paper
1 later
2311.15941 (2023-11)
Tell2Design: A Dataset for Language-Guided Floor Plan Generation
isEnglish 1 other paper
1 later
Liu_RIATIG_Reliable_and_Imperceptible_Adversarial_Text-to-Image_Generation_With_Natural_Prompts_CVPR_2023_paper (2023 (year only))
RIATIG: Reliable and Imperceptible Adversarial Text-to-Image Generation with Natural Prompts

Presence, not direction: an identical function body in two papers' repositories does not say who wrote it, which came from which, or that either paper's method rests on it. Matching is exact after normalisation, so a renamed or edited function is not matched. Dates are when papers appeared, not when code was written. Only .py files the harvester lifted are covered; notebooks are not read. A new-style arXiv id YYMM.NNNN dates a paper to the month 20YY-MM; otherwise Paper.year dates it to the year; otherwise it is undated. Dates are when the paper appeared, not when the code was written. Read 2026-11-22.

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

We introduce GLM-130B, a bilingual (English and Chinese) pre-trained language model with 130 billion parameters. It is an attempt to open-source a 100B-scale model at least as good as GPT-3 (davinci) and unveil how models of such a scale can be successfully pre-trained. Over the course of this effort, we face numerous unexpected technical and engineering challenges, particularly on loss spikes and divergence. In this paper, we introduce the training process of GLM-130B including its design choices, training strategies for both efficiency and stability, and engineering efforts. The resultant GLM-130B model offers significant outperformance over GPT-3 175B (davinci) on a wide range of popular English benchmarks while the performance advantage is not observed in OPT-175B and BLOOM-176B. It also consistently and significantly outperforms ERNIE TITAN 3.0 260B-the largest Chinese language model-across related benchmarks. Finally, we leverage a unique scaling property of GLM-130B to reach INT4 quantization without post training, with almost no performance loss, making it the first among 100B-scale models and more importantly, allowing its effective inference on 4×RTX 3090 (24G) or 8×RTX 2080 Ti (11G) GPUs, the most affordable GPUs required for using 100B-scale models. The GLM-130B model weights are publicly accessible and its code, training logs, related toolkit, and lessons learned are open-sourced at https://github.com/THUDM/GLM-130B/.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2210.02414")
get_code_for_paper("2210.02414")
have("2210.02414")

The run record, dated, one paper per request, free:

curl https://syntology.ai/api/ran/2210.02414.json

A badge for a README (the split and the date, never a ratio):

[![Syntology run record](https://syntology.ai/api/ran/2210.02414.svg)](https://syntology.ai/paper/2210.02414)

Connect an agent — have() is free.