SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2502.08356 · 2025

Systematic Knowledge Injection into Large Language Models via Diverse Augmentation for Domain-Specific RAG

arXiv · PDF · Open in the Atlas

Code that ran

9 of 9 samples (functions and classes) lifted from this paper's linked repositories executed on a synthesized input in Syntology's jail (0 of them constructed an object rather than computing a result; 0 honoured the contract we drafted, 0 violated it and 9 have no contract verdict; 0 failures of our instrument; 0 no recorded run), measured 2026-11-22; repository links from the Papers with Code archive attributed under CC BY-SA, no affiliation; bodies inline only under a permissive licence as GitHub detects it. Not a reproduction of the paper's results.

"Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results. This record, dated, as JSON: /api/ran/2502.08356.json · badge: /api/ran/2502.08356.svg

Each repository's role is a label: a classifier's output recorded on the link, or, where no role is recorded, what the Papers with Code archive marked. The line under it names its basis. Neither says who wrote the code.

RepositoryRoleRan
kushagrabhushan/Systematic-Knowledge-Injection canonical (classifier label)
classifier basis not recorded on the link
9 ran · 0 no recorded run
FunctionStatusWhere it lives
capture_chapter_title Ran
ran on a synthesized input
kushagrabhushan/Systematic-Knowledge-Injection/py_scripts/create_and_index_chunks.py
code served (permissive licence) · get_code("bdbd75911ca8043c")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned str. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import re

def capture_chapter_title(text, chapter_number):
    pattern = re.compile(f'Chapter {chapter_number}\\.\\s*(.*)')
    match = pattern.search(text)
    if match:
        return match.group(1).strip()
    else:
        return None

# Setup code
text = "This is some content.\nChapter 5. Introduction to Advanced Topics\nMore content here."
chapter_number = 5

ARGS = {
    "text": text,
    "chapter_number": chapter_number
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type str — a str output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a str output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
force_retriever_failure Ran
ran on a synthesized input
kushagrabhushan/Systematic-Knowledge-Injection/py_scripts/corrupt_retriever.py
code served (permissive licence) · get_code("104b7adca353b01d")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned list, ndim 1, dtype family int; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import random

class Args:
    def __init__(self, corruption_strategy):
        self.corruption_strategy = corruption_strategy

# Create a row dict with the expected structure
row = {
    'int_id_list': [1, 2, 3, 4, 5],
    'retrieved_ids': [3, 6, 7, 2, 8, 9, 1]
}

# Create args with corruption_strategy
args = Args(corruption_strategy='random')

ARGS = {
    'row': row,
    'args': args
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type list, ndim 1, dtype family int — a list output has no array shape of its own; none is stated
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
get_chapter_text Ran
ran on a synthesized input
kushagrabhushan/Systematic-Knowledge-Injection/py_scripts/create_and_index_chunks.py
code served (permissive licence) · get_code("c2b4bd7f6c40dcdf")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned str. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import re

# Construct a document with multiple chapters formatted as expected
doc = """
# Chapter 1. Introduction section
This is the content of chapter 1.
Some text here.

# Chapter 2. Main content
This is the content of chapter 2.
More interesting material.

# Chapter 3. Advanced topics
This is the content of chapter 3.
Final chapter content.
"""

# Test case 1: chapter exists with next chapter
chapter_1 = 1

# Test case 2: alternative - use a simpler doc for testing chapter extraction
doc_simple = """# Chapter 1. First chapter text
Some content here for chapter 1.

# Chapter 2. Second chapter text
Some content here for chapter 2.

# Chapter 3. Third chapter text
Final content.
"""

# The function takes two parameters: doc and chapter
ARGS = {
    "doc": doc_simple,
    "chapter": 1
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type str — a str output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a str output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
parse_arguments Ran
ran on a synthesized input
kushagrabhushan/Systematic-Knowledge-Injection/py_scripts/assign_gold_passage_ids_to_subdocs.py
code served (permissive licence) · get_code("675b9f9a51334c87")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned Namespace. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import argparse

ARGS = {
    'arg_string': None
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Namespace — a Namespace output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a Namespace output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
parse_arguments Ran
ran on a synthesized input
kushagrabhushan/Systematic-Knowledge-Injection/py_scripts/corrupt_retriever.py
code served (permissive licence) · get_code("b938cda3e98319eb")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned Namespace. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import argparse

def parse_arguments(arg_string=None):
    parser = argparse.ArgumentParser(description=' ')
    parser.add_argument('--input_dir', type=str, help='path to save the datset.')
    parser.add_argument('--output_dir', type=str, help='path to save the datset.')
    parser.add_argument('--corrupt_prob', type=float, default=0)
    parser.add_argument('--single_nodes_file', type=str, default=None)
    parser.add_argument('--text_col_in_single_nodes', type=str, default='document')
    parser.add_argument('--seed', type=int, default=4020)
    parser.add_argument('--corruption_strategy', type=str, choices=['orig', 'reverse', 'random'], default='orig', help='after deleting gold ids, how to order the retrieved ids?')
    parser.add_argument('--retain_existing_corruption_for_train', type=int, default=1, help='input data itself may have retriever failures. Should we keep those intact for train data or erase all initial corruption so that retriever success is mimicked even for them?')
    if arg_string is None:
        args = parser.parse_args()
    else:
        args = parser.parse_args(arg_string.split())
    return args

ARGS = {
    'arg_string': '--input_dir /path/to/input --output_dir /path/to/output --corrupt_prob 0.5 --single_nodes_file nodes.txt --text_col_in_single_nodes document --seed 42 --corruption_strategy random --retain_existing_corruption_for_train 0'
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Namespace — a Namespace output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a Namespace output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
parse_arguments Ran
ran on a synthesized input
kushagrabhushan/Systematic-Knowledge-Injection/py_scripts/create_and_index_chunks.py
code served (permissive licence) · get_code("0ea467579b57a36e")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned Namespace. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import argparse

# The parse_arguments function takes an optional arg_string parameter
# When arg_string is None, it uses sys.argv; when provided, it splits the string into args
# We'll test with both cases by providing an arg_string

arg_string = (
    "--data_path_dir corpus/test_data "
    "--dataset_save_path corpus/output "
    "--chunk_size 256 "
    "--model_name BAAI/bge-small-en-v1.5 "
    "--db_path test_chroma_db "
    "--should_index 1"
)

ARGS = {
    'arg_string': arg_string
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Namespace — a Namespace output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a Namespace output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
parse_arguments Ran
ran on a synthesized input
kushagrabhushan/Systematic-Knowledge-Injection/py_scripts/retrieve_passages_for_synthetic_data.py
code served (permissive licence) · get_code("2da24db55f2ec020")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned Namespace. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import argparse

ARGS = {
    'arg_string': '--db_path ./test_db --model_name BAAI/bge-large-en-v1.5 --top_k 5 --force 0 --qa_dir ./qa_data --output_dir ./output --cache_dir ./cache'
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Namespace — a Namespace output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a Namespace output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
populate_cache Ran
ran on a synthesized input
kushagrabhushan/Systematic-Knowledge-Injection/py_scripts/retrieve_passages_for_synthetic_data.py
code served (permissive licence) · get_code("e1ec4ece60036dfb")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned dict. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import os
import pandas as pd
import tempfile
import json

# Create a temporary directory for cache files
temp_dir = tempfile.mkdtemp()

# Create sample JSONL files for val, test, train splits
for split in ['val', 'test', 'train']:
    fname = os.path.join(temp_dir, f'retrieved_{split}.jsonl')
    data = [
        {
            'question': f'question_{split}_1',
            'retrieved_ids': [1, 2, 3],
            'retrieved_passages': ['passage 1', 'passage 2', 'passage 3']
        },
        {
            'question': f'question_{split}_2',
            'retrieved_ids': [4, 5],
            'retrieved_passages': ['passage 4', 'passage 5']
        }
    ]
    with open(fname, 'w') as f:
        for item in data:
            f.write(json.dumps(item) + '\n')

# Create args object with cache_dir attribute
class Args:
    def __init__(self, cache_dir):
        self.cache_dir = cache_dir

ARGS = {
    'args': Args(cache_dir=temp_dir)
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
pandas — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type dict — a dict output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a dict output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
select_retrieved_ids Ran
ran on a synthesized input
kushagrabhushan/Systematic-Knowledge-Injection/py_scripts/corrupt_retriever.py
code served (permissive licence) · get_code("5e44661cb8ad8f95")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned int, ndim 0, dtype family int, shape () as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import random

# Construct a minimal row dict that satisfies the function's requirements
# The function checks len(row['corrupt_retrieved_ids']) >= topk
# So we need a dict with a 'corrupt_retrieved_ids' key containing a sequence

row = {
    'corrupt_retrieved_ids': [1, 2, 3, 4, 5]  # A list with 5 elements
}

topk = 3  # Less than the length of corrupt_retrieved_ids

corrupt_prob = 0.5  # Probability value between 0 and 1

ARGS = {
    'row': row,
    'topk': topk,
    'corrupt_prob': corrupt_prob
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type int, ndim 0, dtype family int, shape () — shape () as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block

Also present in other papers' code

Of 9 samples lifted from this paper's linked repositories, 9 carry a function identity and 0 do not. Of the 9 distinct functions with an identity, 0 are also present in other papers whose code we hold (0 with at least one paper dated earlier, 0 the same month, 0 later) and 9 are only here among the 52,977 papers whose code Syntology holds (as of 2026-09-26).

Only here means: among the 52,977 papers whose code Syntology holds (51,887 with lifted samples, 1,090 through their assemblies only), as of 2026-09-26, no other paper's lifted samples or current assembly bricks contain an identical function body after normalisation.

Presence, not direction: an identical function body in two papers' repositories does not say who wrote it, which came from which, or that either paper's method rests on it. Matching is exact after normalisation, so a renamed or edited function is not matched. Dates are when papers appeared, not when code was written. Only .py files the harvester lifted are covered; notebooks are not read. A new-style arXiv id YYMM.NNNN dates a paper to the month 20YY-MM; otherwise Paper.year dates it to the year; otherwise it is undated. Dates are when the paper appeared, not when the code was written. Read 2026-11-22.

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

Retrieval-Augmented Generation (RAG) has emerged as a prominent method for incorporating domain knowledge into Large Language Models (LLMs). While RAG enhances response relevance by incorporating retrieved domain knowledge in the context, retrieval errors can still lead to hallucinations and incorrect answers. To recover from retriever failures, domain knowledge is injected by fine-tuning the model to generate the correct response, even in the case of retrieval errors. However, we observe that without systematic knowledge augmentation, fine-tuned LLMs may memorize new information but still fail to extract relevant domain knowledge, leading to poor performance. In this work, we present a novel framework that significantly enhances the fine-tuning process by augmenting the training data in two ways -- context augmentation and knowledge paraphrasing. In context augmentation, we create multiple training samples for a given QA pair by varying the relevance of the retrieved information, teaching the model when to ignore and when to rely on retrieved content. In knowledge paraphrasing, we fine-tune with multiple answers to the same question, enabling LLMs to better internalize specialized knowledge. To mitigate catastrophic forgetting due to fine-tuning, we add a domain-specific identifier to a question and also utilize a replay buffer containing general QA pairs. Experimental results demonstrate the efficacy of our method over existing techniques, achieving up to 10\% relative gain in token-level recall while preserving the LLM's generalization capabilities.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2502.08356")
get_code_for_paper("2502.08356")
have("2502.08356")

The run record, dated, one paper per request, free:

curl https://syntology.ai/api/ran/2502.08356.json

A badge for a README (the split and the date, never a ratio):

[![Syntology run record](https://syntology.ai/api/ran/2502.08356.svg)](https://syntology.ai/paper/2502.08356)

Connect an agent — have() is free.