SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2303.03750 · 2023

Preparing the Vuk'uzenzele and ZA-gov-multilingual South African multilingual corpora

arXiv · PDF · Open in the Atlas

Code that ran

12 of 17 samples (functions and classes) lifted from this paper's linked repositories executed on a synthesized input in Syntology's jail (0 of them constructed an object rather than computing a result; 0 honoured the contract we drafted, 0 violated it and 11 have no contract verdict; 1 failure of our instrument; 5 no recorded run), measured 2026-11-22; repository links from the Papers with Code archive attributed under CC BY-SA, no affiliation; bodies inline only under a permissive licence as GitHub detects it. Not a reproduction of the paper's results.

"Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results. This record, dated, as JSON: /api/ran/2303.03750.json · badge: /api/ran/2303.03750.svg

Each repository's role is a label: a classifier's output recorded on the link, or, where no role is recorded, what the Papers with Code archive marked. The line under it names its basis. Neither says who wrote the code.

RepositoryRoleRan
dsfsi/vukuzenzele-nlp canonical (classifier label)
classifier basis not recorded on the link
11 ran · 5 no recorded run
dsfsi/gov-za-multilingual canonical (classifier label)
classifier basis: classifier_2025
1 ran · 0 no recorded run
FunctionStatusWhere it lives
EncodeTime Ran
ran; our contract draft was wrong, not the code
dsfsi/gov-za-multilingual/src/sentence_alignment/LASER/source/embed.py
code served (permissive licence) · get_code("32b77bc5146315f7")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1) and returned str. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import time

# The function EncodeTime takes a single parameter 't' which should be a numeric value
# representing a past timestamp (seconds since epoch). The function computes the elapsed
# time and formats it as a string.

# We'll create a timestamp from a few seconds ago
past_time = time.time() - 45  # 45 seconds ago

ARGS = {
    't': past_time
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1
Imports
none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type str — a str output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a str output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
LoadData Ran
ran on a synthesized input
dsfsi/vukuzenzele-nlp/src/sentence_alignment/LASER/source/sent_classif.py
code served (permissive licence) · get_code("f379cff248430730")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned DataLoader. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import numpy as np
import torch
import torch.utils.data as data_utils
import tempfile
import os

torch.manual_seed(0)

# Create a temporary directory for test files
tmpdir = tempfile.mkdtemp()

# Create a small binary data file with float32 values
# We'll create data with 8 elements total (2 samples × 4 dimensions)
data = np.array([0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8], dtype=np.float32)
data_file = os.path.join(tmpdir, 'data.bin')
data.tofile(data_file)

# Create a small labels file with 2 labels
labels = np.array([0, 1], dtype=np.int32)
labels_file = os.path.join(tmpdir, 'labels.txt')
np.savetxt(labels_file, labels, fmt='%d')

ARGS = {
    'bdir': tmpdir + '/',
    'dfn': 'data.bin',
    'lfn': 'labels.txt',
    'dim': 4,
    'bsize': 2,
    'shuffle': False,
    'quiet': True
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type DataLoader — a DataLoader output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a DataLoader output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
LoadDataNLI Ran
ran on a synthesized input
dsfsi/vukuzenzele-nlp/src/sentence_alignment/LASER/source/nli.py
code served (permissive licence) · get_code("c831d98b7ae83757")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned DataLoader. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.utils.data as data_utils
import numpy as np
import tempfile
import os

torch.manual_seed(0)

# Create temporary files with binary data
temp_dir = tempfile.mkdtemp()

# Create fn1: binary float32 file with shape (2, 1024) -> 2048 elements
fn1 = os.path.join(temp_dir, "fn1.bin")
x_data = np.random.randn(2, 1024).astype(np.float32)
x_data.tofile(fn1)

# Create fn2: binary float32 file with shape (2, 1024) -> 2048 elements
fn2 = os.path.join(temp_dir, "fn2.bin")
y_data = np.random.randn(2, 1024).astype(np.float32)
y_data.tofile(fn2)

# Create fn_lbl: text file with 2 labels
fn_lbl = os.path.join(temp_dir, "labels.txt")
labels = np.array([0, 1], dtype=np.int32)
np.savetxt(fn_lbl, labels, fmt='%d')

ARGS = {
    'fn1': fn1,
    'fn2': fn2,
    'fn_lbl': fn_lbl,
    'dim': 1024,
    'bsize': 32,
    'fraction': 1.0,
    'shuffle': False,
    'quiet': False,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
faiss, numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type DataLoader — a DataLoader output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a DataLoader output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
TextLoadUnify Ran
ran on a synthesized input
dsfsi/vukuzenzele-nlp/src/sentence_alignment/LASER/source/mine_bitexts.py
code served (permissive licence) · get_code("a9baa7db4816b153")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned tuple. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import tempfile
import os

class Args:
    def __init__(self):
        self.verbose = True
        self.encoding = 'utf-8'
        self.unify = True

# Create a temporary file with some text content
temp_file = tempfile.NamedTemporaryFile(mode='w', delete=False, suffix='.txt', encoding='utf-8')
temp_file.write("hello world\n")
temp_file.write("foo bar\n")
temp_file.write("hello world\n")
temp_file.write("baz qux\n")
temp_file.close()

fname = temp_file.name

args = Args()

ARGS = {
    'fname': fname,
    'args': args
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type tuple — a tuple output has no array shape of its own; none is stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
align Ran
ran on a synthesized input
dsfsi/vukuzenzele-nlp/src/sentence_alignment/sentence_alignment.py
code served (permissive licence) · get_code("50880fa89b73baca")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned list, ndim 1, dtype family float; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import numpy as np

# Analyze the function signature and body:
# align(src, tgt, vectors, tokens)
# - src: string key for source language
# - tgt: string key for target language
# - vectors: tuple of (src_vect, tgt_vect) where each is a list of arrays
# - tokens: tuple of (src_tokens, tgt_tokens) where each is a list of lists

# Create small sentence vectors (2D arrays, each representing a sentence embedding)
src_vectors = [np.array([1.0, 0.1, 0.2, 0.05]), np.array([0.9, 0.15, 0.18, 0.08])]
tgt_vectors = [np.array([0.95, 0.12, 0.19, 0.06]), np.array([0.85, 0.2, 0.25, 0.1])]

# Create token lists (each sentence has a list of tokens)
src_tokens = [['hello', 'world'], ['how', 'are', 'you']]
tgt_tokens = [['hola', 'mundo'], ['como', 'estas']]

# Arguments for the align function
ARGS = {
    'src': 'en',
    'tgt': 'es',
    'vectors': (src_vectors, tgt_vectors),
    'tokens': (src_tokens, tgt_tokens)
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
numpy, sklearn — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type list, ndim 1, dtype family float — a list output has no array shape of its own; none is stated
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
build_date Ran
ran on a synthesized input
dsfsi/vukuzenzele-nlp/src/sentence_alignment/file_handler.py
code served (permissive licence) · get_code("003bb0e63b4d6e7f")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned date. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import os
import re
import json
import pandas
from datetime import date

ARGS = {
    'edition': '2020-01-ed2'
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
pandas — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type date — a date output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a date output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
calculate_error Ran
ran on a synthesized input
dsfsi/vukuzenzele-nlp/src/sentence_alignment/LASER/source/xsim.py
code served (permissive licence) · get_code("5515760c7736c79f")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned tuple, ndim 1, dtype family int; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import numpy as np
import tempfile
import os

# Set seed for reproducibility
np.random.seed(0)

# Create small test data
nbex = 2  # number of examples
dim = 4   # dimensionality

# Generate random normalized vectors
x = np.random.randn(nbex, dim).astype(np.float32)
y = np.random.randn(nbex, dim).astype(np.float32)

# Normalize them (since the function uses faiss.normalize_L2)
x = x / (np.linalg.norm(x, axis=1, keepdims=True) + 1e-8)
y = y / (np.linalg.norm(y, axis=1, keepdims=True) + 1e-8)

# Create a temporary eval_text file with matching lines
temp_dir = tempfile.gettempdir()
eval_file = os.path.join(temp_dir, "test_eval.txt")
with open(eval_file, 'w', encoding='utf-8') as f:
    f.write("line 0\n")
    f.write("line 1\n")

ARGS = {
    'x': x,
    'y': y,
    'margin': 'ratio',
    'k': 2,
    'eval_text': eval_file
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
faiss, numpy — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type tuple, ndim 1, dtype family int — a tuple output has no array shape of its own; none is stated
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
cosine_score Ran
ran on a synthesized input
dsfsi/vukuzenzele-nlp/src/sentence_alignment/sentence_alignment.py
code served (permissive licence) · get_code("7b360a8e774ee561")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned float32, ndim 0, dtype family float, shape () as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import numpy as np
import torch

torch.manual_seed(0)

# Create small sentence vectors as numpy arrays
# The function calls .reshape(1, -1) on both, so they should be 1D arrays
src = np.random.randn(4).astype(np.float32)
tgt = np.random.randn(4).astype(np.float32)

ARGS = {
    "src": src,
    "tgt": tgt,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
numpy, sklearn, torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type float32, ndim 0, dtype family float, shape () — shape () as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
fix_end_of_sentence Ran
ran on a synthesized input
dsfsi/vukuzenzele-nlp/src/sentence_alignment/sentence_tokenisation.py
code served (permissive licence) · get_code("0d43ad3da9c8d319")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned str. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import nltk
import re
import os

# The function fix_end_of_sentence takes a single parameter: text (str)
# It processes the text to add spaces after sentence-ending punctuation
# when followed by lowercase letters, with exceptions for domain suffixes like .co.za

ARGS = {
    "text": "This is a sentence.Another sentence here! And a question?No space after this.Also check example.co.za domain names."
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
nltk — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type str — a str output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a str output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
knn Ran
ran on a synthesized input
dsfsi/vukuzenzele-nlp/src/sentence_alignment/LASER/source/mine_bitexts.py
code served (permissive licence) · get_code("facdcfc340c5dd83")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned tuple, ndim 3, dtype family float; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import numpy as np

np.random.seed(0)

# Create small dummy data for x and y
# x: (2, 4) - 2 samples, 4 dimensions
# y: (3, 4) - 3 samples, 4 dimensions
x = np.random.randn(2, 4).astype(np.float32)
y = np.random.randn(3, 4).astype(np.float32)

k = 2
use_gpu = False

ARGS = {
    "x": x,
    "y": y,
    "k": k,
    "use_gpu": use_gpu
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
faiss, numpy — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type tuple, ndim 3, dtype family float — a tuple output has no array shape of its own; none is stated
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
pre_process_text Ran
ran on a synthesized input
dsfsi/vukuzenzele-nlp/src/sentence_alignment/sentence_tokenisation.py
code served (permissive licence) · get_code("5eed8aba4367439f")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned str. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import nltk
import re
import os

def fix_end_of_sentence(text: str):
    """
  ### Adds a space after the end of a sentence without a space
  #### Example
  The tokeniser will interpret this as a whole sentence.A human reader can detect the missing space.

  It also doesn't make .co.za & co.za split up because that would be lame.
  """
    end_of_sentece_chars = ['.', '?', '!']
    i = 0
    while i < len(text) - 1:
        if text[i] in end_of_sentece_chars:
            if ord(text[i + 1]) >= 97 and ord(text[i + 1]) <= 122:
                if re.search('[a-z]\\.co\\.za', text[i - 1:i + 6]) or re.search('\\.co\\.za', text[i - 3:i + 3]):
                    i += 1
                    continue
                text = ' '.join([text[:i + 1], text[i + 1:]])
        i += 1
    return text

def pre_process_text(text):
    text = text.lower()
    text = re.sub('\\-', '', text)
    text = re.sub('\\"', '"', text)
    text = re.sub('\\"', '"', text)
    text = re.sub('\\•', '', text)
    text = re.sub('ʼn', 'n', text)
    text = re.sub('\\s{2,}', ' ', text)
    text = re.sub('\t', ' ', text)
    text = re.sub('\n', ' ', text)
    text = fix_end_of_sentence(text)
    return text

ARGS = {
    "text": "Hello world.This is a test! Another sentence? And more text with hyphens-and-quotes.Also test with.co.za domain names.co.za should work fine."
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
nltk — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type str — a str output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a str output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
update_indices Ran
ran on a synthesized input
dsfsi/vukuzenzele-nlp/src/sentence_alignment/sentence_alignment.py
code served (permissive licence) · get_code("f31d97d8fa45ac21")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned tuple, ndim 1, dtype family int; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import numpy as np

torch.manual_seed(0)

# Create small sentence vectors (embeddings)
# src_vect: 4 sentences, each with 8-dimensional embedding
src_vect = [
    np.random.randn(8).astype(np.float32),
    np.random.randn(8).astype(np.float32),
    np.random.randn(8).astype(np.float32),
    np.random.randn(8).astype(np.float32),
]

# tgt_vect: 3 sentences, each with 8-dimensional embedding
tgt_vect = [
    np.random.randn(8).astype(np.float32),
    np.random.randn(8).astype(np.float32),
    np.random.randn(8).astype(np.float32),
]

# Create the vectors tuple
vectors = (src_vect, tgt_vect)

# Create starting indices
indexes = (0, 0)

ARGS = {
    "indexes": indexes,
    "vectors": vectors,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
numpy, sklearn, torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type tuple, ndim 1, dtype family int — a tuple output has no array shape of its own; none is stated
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
fetch_data_edition_filepaths Not yet run
unverified (harvested, no recorded run)
dsfsi/vukuzenzele-nlp/src/sentence_alignment/file_handler.py
code served (permissive licence) · get_code("1c1f0aab1161ba49")
How it ran

no invocation recorded.

fetch_data_txt_filepaths Not yet run
unverified (harvested, no recorded run)
dsfsi/vukuzenzele-nlp/src/sentence_alignment/file_handler.py
code served (permissive licence) · get_code("d33cd6edb59a4ab3")
How it ran

no invocation recorded.

knnGPU Not yet run
unverified (harvested, no recorded run)
dsfsi/vukuzenzele-nlp/src/sentence_alignment/LASER/source/mine_bitexts.py
code served (permissive licence) · get_code("05395b64953253bc")
How it ran

no invocation recorded.

tokenise Not yet run
unverified (harvested, no recorded run)
dsfsi/vukuzenzele-nlp/src/sentence_alignment/sentence_tokenisation.py
code served (permissive licence) · get_code("f465d355228a688f")
How it ran

no invocation recorded.

xSIM Not yet run
unverified (harvested, no recorded run)
dsfsi/vukuzenzele-nlp/src/sentence_alignment/LASER/source/xsim.py
code served (permissive licence) · get_code("2eb5f73b0eda01c2")
How it ran

no invocation recorded.

Also present in other papers' code

Of 17 samples lifted from this paper's linked repositories, 17 carry a function identity and 0 do not. Of the 17 distinct functions with an identity, 0 are also present in other papers whose code we hold (0 with at least one paper dated earlier, 0 the same month, 0 later) and 17 are only here among the 52,977 papers whose code Syntology holds (as of 2026-09-26).

Only here means: among the 52,977 papers whose code Syntology holds (51,887 with lifted samples, 1,090 through their assemblies only), as of 2026-09-26, no other paper's lifted samples or current assembly bricks contain an identical function body after normalisation.

Presence, not direction: an identical function body in two papers' repositories does not say who wrote it, which came from which, or that either paper's method rests on it. Matching is exact after normalisation, so a renamed or edited function is not matched. Dates are when papers appeared, not when code was written. Only .py files the harvester lifted are covered; notebooks are not read. A new-style arXiv id YYMM.NNNN dates a paper to the month 20YY-MM; otherwise Paper.year dates it to the year; otherwise it is undated. Dates are when the paper appeared, not when the code was written. Read 2026-11-22.

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

This paper introduces two multilingual government themed corpora in various South African languages. The corpora were collected by gathering the South African Government newspaper (Vuk'uzenzele), as well as South African government speeches (ZA-gov-multilingual), that are translated into all 11 South African official languages. The corpora can be used for a myriad of downstream NLP tasks. The corpora were created to allow researchers to study the language used in South African government publications, with a focus on understanding how South African government officials communicate with their constituents. In this paper we highlight the process of gathering, cleaning and making available the corpora. We create parallel sentence corpora for Neural Machine Translation (NMT) tasks using Language-Agnostic Sentence Representations (LASER) embeddings. With these aligned sentences we then provide NMT benchmarks for 9 indigenous languages by fine-tuning a massively multilingual pre-trained language model.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2303.03750")
get_code_for_paper("2303.03750")
have("2303.03750")

The run record, dated, one paper per request, free:

curl https://syntology.ai/api/ran/2303.03750.json

A badge for a README (the split and the date, never a ratio):

[![Syntology run record](https://syntology.ai/api/ran/2303.03750.svg)](https://syntology.ai/paper/2303.03750)

Connect an agent — have() is free.