| Ngrams |
Ran ran on a synthesized input · constructed an object; no result was computed |
pltrdy/rouge/rouge/rouge_score.py code served (permissive licence) · get_code("0b1b6496ee19d475") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, Ngrams: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import numpy as np
from copy import deepcopy
torch.manual_seed(0)
# The Ngrams class constructor takes:
# - ngrams: a collection (set or list) of ngram objects
# - exclusive: boolean flag
# Looking at how the class is used:
# - ngrams parameter is passed to set() or list() constructor
# - Methods like intersection() and union() work with ._ngrams attribute
# - The class stores tuples or hashable objects as ngrams
# Create sample ngrams (using tuples as they're hashable and work with both set/list)
ngrams_data_exclusive = {("the", "cat"), ("cat", "sat"), ("sat", "on")}
ngrams_data_non_exclusive = [("the", "cat"), ("cat", "sat"), ("sat", "on"), ("cat", "sat")]
ARGS = {
"ngrams": ngrams_data_exclusive,
"exclusive": True
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type Ngrams — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a Ngrams output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| _get_ngrams |
Ran ran; our contract draft was wrong, not the code |
pltrdy/rouge/rouge/rouge_score.py code served (permissive licence) · get_code("699442d33ab83bd2") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned Ngrams. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
from copy import deepcopy
class Ngrams(object):
"""
Ngrams datastructure based on `set` or `list`
depending in `exclusive`
"""
def __init__(self, ngrams={}, exclusive=True):
if exclusive:
self._ngrams = set(ngrams)
else:
self._ngrams = list(ngrams)
self.exclusive = exclusive
def add(self, o):
if self.exclusive:
self._ngrams.add(o)
else:
self._ngrams.append(o)
def __len__(self):
return len(self._ngrams)
def intersection(self, o):
if self.exclusive:
inter_set = self._ngrams.intersection(o._ngrams)
return Ngrams(inter_set, exclusive=True)
else:
other_list = deepcopy(o._ngrams)
inter_list = []
for e in self._ngrams:
try:
i = other_list.index(e)
except ValueError:
continue
other_list.pop(i)
inter_list.append(e)
return Ngrams(inter_list, exclusive=False)
def union(self, *ngrams):
if self.exclusive:
union_set = self._ngrams
for o in ngrams:
union_set = union_set.union(o._ngrams)
return Ngrams(union_set, exclusive=True)
else:
union_list = deepcopy(self._ngrams)
for o in ngrams:
union_list.extend(o._ngrams)
return Ngrams(union_list, exclusive=False)
def _get_ngrams(n, text, exclusive=True):
"""Calcualtes n-grams.
Args:
n: which n-grams to calculate
text: An array of tokens
Returns:
A set of n-grams
"""
ngram_set = Ngrams(exclusive=exclusive)
text_length = len(text)
max_index_ngram_start = text_length - n
for i in range(max_index_ngram_start + 1):
ngram_set.add(tuple(text[i:i + n]))
return ngram_set
ARGS = {
"n": 2,
"text": ["hello", "world", "foo", "bar"],
"exclusive": True
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type Ngrams — a Ngrams output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a Ngrams output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| _get_word_ngrams |
Ran ran; our contract draft was wrong, not the code |
pltrdy/rouge/rouge/rouge_score.py code served (permissive licence) · get_code("82942ade3a066704") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned Ngrams. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import itertools
from copy import deepcopy
class Ngrams(object):
"""
Ngrams datastructure based on `set` or `list`
depending in `exclusive`
"""
def __init__(self, ngrams={}, exclusive=True):
if exclusive:
self._ngrams = set(ngrams)
else:
self._ngrams = list(ngrams)
self.exclusive = exclusive
def add(self, o):
if self.exclusive:
self._ngrams.add(o)
else:
self._ngrams.append(o)
def __len__(self):
return len(self._ngrams)
def intersection(self, o):
if self.exclusive:
inter_set = self._ngrams.intersection(o._ngrams)
return Ngrams(inter_set, exclusive=True)
else:
other_list = deepcopy(o._ngrams)
inter_list = []
for e in self._ngrams:
try:
i = other_list.index(e)
except ValueError:
continue
other_list.pop(i)
inter_list.append(e)
return Ngrams(inter_list, exclusive=False)
def union(self, *ngrams):
if self.exclusive:
union_set = self._ngrams
for o in ngrams:
union_set = union_set.union(o._ngrams)
return Ngrams(union_set, exclusive=True)
else:
union_list = deepcopy(self._ngrams)
for o in ngrams:
union_list.extend(o._ngrams)
return Ngrams(union_list, exclusive=False)
def _get_ngrams(n, text, exclusive=True):
"""Calcualtes n-grams.
Args:
n: which n-grams to calculate
text: An array of tokens
Returns:
A set of n-grams
"""
ngram_set = Ngrams(exclusive=exclusive)
text_length = len(text)
max_index_ngram_start = text_length - n
for i in range(max_index_ngram_start + 1):
ngram_set.add(tuple(text[i:i + n]))
return ngram_set
def _split_into_words(sentences):
"""Splits multiple sentences into words and flattens the result"""
return list(itertools.chain(*[_.split(' ') for _ in sentences]))
def _get_word_ngrams(n, sentences, exclusive=True):
"""Calculates word n-grams for multiple sentences.
"""
assert len(sentences) > 0
assert n > 0
words = _split_into_words(sentences)
return _get_ngrams(n, words, exclusive=exclusive)
ARGS = {
"n": 2,
"sentences": ["hello world test", "foo bar baz"],
"exclusive": True
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type Ngrams — a Ngrams output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a Ngrams output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| _split_into_words |
Ran ran; our contract draft was wrong, not the code |
pltrdy/rouge/rouge/rouge_score.py code served (permissive licence) · get_code("6a189ad1c6a91653") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned list, ndim 1, dtype family float; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import itertools
def _split_into_words(sentences):
"""Splits multiple sentences into words and flattens the result"""
return list(itertools.chain(*[_.split(' ') for _ in sentences]))
ARGS = {
'sentences': ['hello world', 'foo bar baz', 'test']
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type list, ndim 1, dtype family float — a list output has no array shape of its own; none is stated
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| clean_text |
Ran ran on a synthesized input |
webis-de/summary-explorer/text-processing/src/utils.py code served (permissive licence) · get_code("f7918be2e65b31a7") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned str. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
# The function clean_text takes a single parameter 'text' which should be a string
# It performs various string replacements on the input
# Create a sample text with various tokens and patterns that the function handles
sample_text = "This is a test <t>with</t> some -lrb- brackets -rrb- and n't contractions and ' quotes and \" double quotes and , commas and . periods and ( parentheses ) here </t> <n>more</n> -LRB- uppercase -RRB- and <s>tags</s>"
ARGS = {
"text": sample_text
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type str — a str output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a str output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| detokenize_relation |
Ran ran on a synthesized input |
webis-de/summary-explorer/text-processing/src/utils.py code served (permissive licence) · get_code("4b2ce6bb44dca364") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned dict. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
def detokenize_text(text):
return text.replace(" n't", "n't").replace(" '", "'").replace(' "', '"').replace(' ,', ',').replace('` ', '`').replace(' .', '.').replace('( ', '(').replace(' )', ')')
def detokenize_relation(relation):
return {k: detokenize_text(v) for k, v in relation.items()}
# Construct a sample relation dict with tokenized text values
relation = {
"subject": "The quick brown fox ( jumps )",
"predicate": "jumps over",
"object": "the lazy dog .",
"context": "In a story , there was n't a better moment ."
}
ARGS = {
"relation": relation
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type dict — a dict output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a dict output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| detokenize_text |
Ran ran on a synthesized input |
webis-de/summary-explorer/text-processing/src/utils.py code served (permissive licence) · get_code("962bbc77ef68d643") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned str. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
def detokenize_text(text):
return text.replace(" n't", "n't").replace(" '", "'").replace(' "', '"').replace(' ,', ',').replace('` ', '`').replace(' .', '.').replace('( ', '(').replace(' )', ')')
ARGS = {
"text": "This is a test . It has ( some ) punctuation , n't and ' quotes with ` backticks ."
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type str — a str output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a str output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| f_r_p_rouge_n |
Ran ran; our contract draft was wrong, not the code |
pltrdy/rouge/rouge/rouge_score.py code served (permissive licence) · get_code("6f967ee0bce679ea") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned dict. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
ARGS = {
'evaluated_count': 10,
'reference_count': 8,
'overlapping_count': 6
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type dict — a dict output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a dict output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| get_entity_counts |
Ran ran on a synthesized input |
webis-de/summary-explorer/text-processing/src/step_1_nlp_pipeline.py code served (permissive licence) · get_code("a9dd30c56406efb3") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned list, ndim 1, dtype family float; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
from itertools import chain
# Construct a minimal processed_record that satisfies the function's requirements
# The function expects:
# - processed_record['sentences'] to be a list of dicts
# - each sentence dict to have an 'entities' key
# - each entity to be a tuple/list of (text, type)
processed_record = {
'sentences': [
{
'entities': [
('Apple', 'ORG'),
('Steve Jobs', 'PERSON'),
]
},
{
'entities': [
('California', 'LOC'),
('Apple', 'ORG'),
]
},
{
'entities': [] # Empty entities list to test filtering
},
{
'entities': [
('Steve Jobs', 'PERSON'),
]
},
]
}
ARGS = {
'processed_record': processed_record
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type list, ndim 1, dtype family float — a list output has no array shape of its own; none is stated
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| get_remaining_content |
Ran ran on a synthesized input |
webis-de/summary-explorer/text-processing/src/step_2_lexical_alignment.py code served (permissive licence) · get_code("3b4c00097055bedb") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned str. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
ARGS = {
"candidate": "The quick brown fox jumps over the lazy dog",
"matched": "The quick fox"
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type str — a str output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a str output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| rouge_n |
Ran ran; our contract draft was wrong, not the code |
pltrdy/rouge/rouge/rouge_score.py code served (permissive licence) · get_code("dd3dfb07f3d3bf48") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1) and returned dict. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import itertools
from copy import deepcopy
# Define ARGS dict with all parameters for rouge_n function
ARGS = {
'evaluated_sentences': ['this is a test', 'another sentence here'],
'reference_sentences': ['this is a test', 'yet another sentence'],
'n': 2,
'raw_results': False,
'exclusive': True
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1
- Imports
- none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type dict — a dict output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a dict output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| bert_score_similarity |
Not yet run unverified (harvested, no recorded run) |
webis-de/summary-explorer/text-processing/src/step_3_semantic_alignment.py code served (permissive licence) · get_code("a7aa528d3a102206") How it ran |
| compute_bert_score |
Not yet run unverified (harvested, no recorded run) |
webis-de/summary-explorer/text-processing/src/metrics.py code served (permissive licence) · get_code("71936af5d44993bd") How it ran |
| compute_mean_rouge_overlap |
Not yet run unverified (harvested, no recorded run) |
webis-de/summary-explorer/text-processing/src/metrics.py code served (permissive licence) · get_code("6cb2e57033eb24ca") How it ran |
| compute_rouge_score |
Not yet run unverified (harvested, no recorded run) |
webis-de/summary-explorer/text-processing/src/metrics.py code served (permissive licence) · get_code("1b84d2e60b85384e") How it ran |