SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2403.10088 · NAACL · 2024

Intent-conditioned and Non-toxic Counterspeech Generation using Multi-Task Instruction Tuning with RLAIF

Tanmoy Chakraborty, Iiit Delhi, Shad Akhtar, Amey Hengle, Sahajpreet Singh, Aswini Kumar, Anil Bandhakavi, Logically Ai

arXiv · PDF · Open in the Atlas

Code that ran

9 of 10 samples (functions and classes) lifted from this paper's linked repositories executed on a synthesized input in Syntology's jail (0 of them constructed an object rather than computing a result; 0 honoured the contract we drafted, 4 violated it and 0 have no contract verdict; 5 failures of our instrument; 1 no recorded run), measured 2026-11-22; bodies inline only under a permissive licence as GitHub detects it. Not a reproduction of the paper's results.

"Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results. This record, dated, as JSON: /api/ran/2403.10088.json · badge: /api/ran/2403.10088.svg

Each repository's role is a label: a classifier's output recorded on the link, or, where no role is recorded, what the Papers with Code archive marked. The line under it names its basis. Neither says who wrote the code.

RepositoryRoleRan
lvwerra/trl canonical (classifier label)
classifier basis: census
9 ran · 1 no recorded run
FunctionStatusWhere it lives
get_gamma_weights Ran
ran; our contract draft was wrong, not the code
lvwerra/trl/trl/losses/grpo_loss.py
code served (permissive licence) · get_code("6fffb84514770d3e")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned Tensor, ndim 2, dtype family float, shape (2, 1) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import math

torch.manual_seed(0)

# Construct tiny tensors for the function
batch_size = 2
seq_len = 4

# advantages: (B,) tensor indicating whether advantages are positive or negative
advantages = torch.randn(batch_size)

# log_ratio_per_token: (B, seq_len) log probability ratios per token
log_ratio_per_token = torch.randn(batch_size, seq_len)

# mask: (B, seq_len) binary mask for valid tokens
mask = torch.ones(batch_size, seq_len)
mask[0, 2:] = 0  # Mask out some tokens for first batch

# importance_sampling_ratio: optional (B, seq_len) tensor of importance weights
importance_sampling_ratio = torch.abs(torch.randn(batch_size, seq_len)) + 0.5

# Hyperparameters
k_pos = 2.0
lambda_pos = 3.0
k_neg = 3.0
lambda_neg = 2.0

ARGS = {
    "advantages": advantages,
    "log_ratio_per_token": log_ratio_per_token,
    "mask": mask,
    "importance_sampling_ratio": importance_sampling_ratio,
    "k_pos": k_pos,
    "lambda_pos": lambda_pos,
    "k_neg": k_neg,
    "lambda_neg": lambda_neg,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Tensor, ndim 2, dtype family float, shape (2, 1) — shape (2, 1) as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
has_generation_markers Ran
ran, violated the contract we drafted
lvwerra/trl/trl/chat_template_utils.py
code served (permissive licence) · get_code("dc8b4f56c93e15a4")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned bool, ndim 0, dtype family float, shape () as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import re

def has_generation_markers(chat_template: str) -> bool:
    """
    Check whether the chat template defines `{% generation %}` markers, accounting for whitespace-trim variants such as
    `{%- generation %}` and `{%- generation -%}`.
    """
    return re.search('\\{%-?\\s*generation\\s*-?%\\}', chat_template) is not None

# Test cases for the chat_template parameter
# The function searches for generation markers with optional whitespace and dash variants

ARGS = {
    "chat_template": "{% generation %}"
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type bool, ndim 0, dtype family float, shape () — shape () as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
is_conversational Ran
ran, violated the contract we drafted
lvwerra/trl/trl/data_utils.py
code served (permissive licence) · get_code("92674d32e9e8fe15")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned bool, ndim 0, dtype family float, shape () as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import numpy as np

# Example 1: Conversational format with messages list containing dict with 'role' key
example_conversational = {
    "prompt": [
        {"role": "user", "content": "What color is the sky?"},
        {"role": "assistant", "content": "The sky is blue."}
    ]
}

# Example 2: Non-conversational format with string prompt
example_non_conversational = {
    "prompt": "The sky is"
}

# Example 3: Conversational format with 'messages' key
example_messages_conversational = {
    "messages": [
        {"role": "user", "content": "Hello"},
        {"role": "assistant", "content": "Hi there"}
    ]
}

# Example 4: Conversational format with 'chosen' key
example_chosen_conversational = {
    "chosen": [
        {"role": "user", "content": "How are you?"},
        {"role": "assistant", "content": "I'm doing well!"}
    ]
}

# Example 5: Non-conversational with list but no role
example_list_no_role = {
    "prompt": ["just a string in a list"]
}

ARGS = {
    "example": example_conversational
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type bool, ndim 0, dtype family float, shape () — shape () as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
is_liger_kernel_available Ran
ran, violated the contract we drafted
lvwerra/trl/trl/import_utils.py
code served (permissive licence) · get_code("27029757ccbfdfa1")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned bool, ndim 0, dtype family float, shape () as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import importlib.util
import importlib.metadata
from packaging.version import Version

LIGER_KERNEL_MIN_VERSION = '0.8.2'

PACKAGE_DISTRIBUTION_MAPPING = importlib.metadata.packages_distributions()

def _is_package_available(pkg_name: str, return_version: bool=False) -> tuple[bool, str] | bool:
    """Check if `pkg_name` exist, and optionally try to get its version"""
    spec = importlib.util.find_spec(pkg_name)
    package_exists = spec is not None
    package_version = 'N/A'
    if package_exists and return_version:
        try:
            distributions = PACKAGE_DISTRIBUTION_MAPPING[pkg_name]
            normalized_pkg_name = pkg_name.replace('_', '-')
            if normalized_pkg_name in distributions:
                distribution_name = normalized_pkg_name
            elif pkg_name in distributions:
                distribution_name = pkg_name
            else:
                distribution_name = distributions[0]
            package_version = importlib.metadata.version(distribution_name)
        except (importlib.metadata.PackageNotFoundError, KeyError):
            package = importlib.import_module(pkg_name)
            package_version = getattr(package, '__version__', 'N/A')
    if return_version:
        return (package_exists, package_version)
    else:
        return package_exists

def is_liger_kernel_available(min_version: str=LIGER_KERNEL_MIN_VERSION) -> bool:
    _liger_kernel_available, _liger_kernel_version = _is_package_available('liger_kernel', return_version=True)
    return _liger_kernel_available and Version(_liger_kernel_version) >= Version(min_version)

ARGS = {
    'min_version': '0.8.2'
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
packaging — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type bool, ndim 0, dtype family float, shape () — shape () as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
is_vllm_available Ran
ran, violated the contract we drafted
lvwerra/trl/trl/import_utils.py
code served (permissive licence) · get_code("bea49da00dcdd049")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned bool, ndim 0, dtype family float, shape () as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import importlib.metadata
import importlib.util
import warnings
from packaging.version import Version

# Mock the PACKAGE_DISTRIBUTION_MAPPING if needed
PACKAGE_DISTRIBUTION_MAPPING = importlib.metadata.packages_distributions()

def _is_package_available(pkg_name: str, return_version: bool=False) -> tuple[bool, str] | bool:
    """Check if `pkg_name` exist, and optionally try to get its version"""
    spec = importlib.util.find_spec(pkg_name)
    package_exists = spec is not None
    package_version = 'N/A'
    if package_exists and return_version:
        try:
            distributions = PACKAGE_DISTRIBUTION_MAPPING[pkg_name]
            normalized_pkg_name = pkg_name.replace('_', '-')
            if normalized_pkg_name in distributions:
                distribution_name = normalized_pkg_name
            elif pkg_name in distributions:
                distribution_name = pkg_name
            else:
                distribution_name = distributions[0]
            package_version = importlib.metadata.version(distribution_name)
        except (importlib.metadata.PackageNotFoundError, KeyError):
            package = importlib.import_module(pkg_name)
            package_version = getattr(package, '__version__', 'N/A')
    if return_version:
        return (package_exists, package_version)
    else:
        return package_exists

def is_vllm_available(min_version: str | None=None) -> bool:
    _vllm_available, _vllm_version = _is_package_available('vllm', return_version=True)
    if _vllm_available:
        _vllm_base_version = Version(Version(_vllm_version).base_version)
        if not Version('0.19.1') <= _vllm_base_version <= Version('0.29.0'):
            warnings.warn(f'TRL currently supports vLLM versions from 0.19.1 to 0.29.0. You have version {_vllm_version} installed. We recommend installing a supported version to avoid compatibility issues.', stacklevel=2)
        if min_version is not None and Version(_vllm_version) < Version(min_version):
            return False
    return _vllm_available

# Setup arguments for is_vllm_available
ARGS = {
    'min_version': None,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
packaging — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type bool, ndim 0, dtype family float, shape () — shape () as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
k3_loss_fn Ran
ran; our contract draft was wrong, not the code
lvwerra/trl/trl/losses/grpo_loss.py
code served (permissive licence) · get_code("86d6504cba39cd87")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned Tensor, ndim 2, dtype family float, shape (2, 4) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch

torch.manual_seed(0)

# k3_loss_fn takes two log probability tensors and computes a KL divergence-like loss
# Both log_p and log_q should be tensors that support arithmetic operations
log_p = torch.randn(2, 4)
log_q = torch.randn(2, 4)

ARGS = {
    "log_p": log_p,
    "log_q": log_q,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Tensor, ndim 2, dtype family float, shape (2, 4) — shape (2, 4) as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
prepare_multimodal_messages Ran
ran; the contract check could not run (RAISES, DEP_MISSING or TIMEOUT)
lvwerra/trl/trl/data_utils.py
code served (permissive licence) · get_code("db7856b1458fa1c3")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned list, ndim 1, dtype family float; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
from PIL import Image
import numpy as np

# Create a small PIL Image for testing
def create_dummy_image():
    arr = np.zeros((4, 4, 3), dtype=np.uint8)
    return Image.fromarray(arr)

# Construct the arguments
ARGS = {
    "messages": [
        {
            "role": "user",
            "content": "What's in this image?"
        },
        {
            "role": "assistant",
            "content": "It looks like a cat."
        }
    ],
    "images": [create_dummy_image()]
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
PIL, numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type list, ndim 1, dtype family float — a list output has no array shape of its own; none is stated
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
prepare_multimodal_messages_vllm Ran
ran; our contract draft was wrong, not the code
lvwerra/trl/trl/data_utils.py
code served (permissive licence) · get_code("631e70d7b6a27f0f")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned list, ndim 1, dtype family float; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

from PIL import Image
import io

# Create a small PIL Image for testing
def create_test_image():
    img = Image.new('RGB', (10, 10), color='red')
    return img

# Construct valid arguments for prepare_multimodal_messages_vllm
ARGS = {
    "messages": [
        {
            "role": "user",
            "content": [
                {
                    "type": "image",
                    "image": create_test_image()
                },
                {
                    "type": "text",
                    "text": "What's in this image?"
                }
            ]
        },
        {
            "role": "assistant",
            "content": [
                {
                    "type": "text",
                    "text": "This is a red square."
                }
            ]
        }
    ]
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
PIL — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type list, ndim 1, dtype family float — a list output has no array shape of its own; none is stated
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
sapo_loss_fn Ran
ran; the contract check could not run (RAISES, DEP_MISSING or TIMEOUT)
lvwerra/trl/trl/losses/grpo_loss.py
code served (permissive licence) · get_code("aa492ab4bceb5b94")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned Tensor, ndim 1, dtype family float, shape (4,) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch

torch.manual_seed(0)

# importance_ratio: torch.Tensor - a tensor of importance sampling ratios (pi_theta / pi_old)
# The function performs element-wise operations on it
importance_ratio = torch.tensor([0.8, 1.0, 1.2, 1.5], dtype=torch.float32)

# temperature: float - a positive temperature parameter controlling smoothness
temperature = 0.5

ARGS = {
    "importance_ratio": importance_ratio,
    "temperature": temperature,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Tensor, ndim 1, dtype family float, shape (4,) — shape (4,) as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
clone_chat_template Not yet run
unverified (harvested, no recorded run)
lvwerra/trl/trl/chat_template_utils.py
code served (permissive licence) · get_code("cb0eb39dbaf96b03")
How it ran

no invocation recorded.

Also present in other papers' code

Of 10 samples lifted from this paper's linked repositories, 10 carry a function identity and 0 do not. Of the 10 distinct functions with an identity, 3 are also present in other papers whose code we hold (0 with at least one paper dated earlier, 0 the same month, 3 later) and 7 are only here among the 52,977 papers whose code Syntology holds (as of 2026-09-26).

Only here means: among the 52,977 papers whose code Syntology holds (51,887 with lifted samples, 1,090 through their assemblies only), as of 2026-09-26, no other paper's lifted samples or current assembly bricks contain an identical function body after normalisation.

FunctionAlso present inEarliest-datedOthers named
is_conversational 1 other paper
1 later
2606.29712 (2026-06)
Why Struggle with Continuous Latents? Interpretable Discrete Latent Reasoning via Rendered Compression
is_liger_kernel_available 1 other paper
1 later
2606.29712 (2026-06)
Why Struggle with Continuous Latents? Interpretable Discrete Latent Reasoning via Rendered Compression
prepare_multimodal_messages_vllm 1 other paper
1 later
2606.29712 (2026-06)
Why Struggle with Continuous Latents? Interpretable Discrete Latent Reasoning via Rendered Compression

Presence, not direction: an identical function body in two papers' repositories does not say who wrote it, which came from which, or that either paper's method rests on it. Matching is exact after normalisation, so a renamed or edited function is not matched. Dates are when papers appeared, not when code was written. Only .py files the harvester lifted are covered; notebooks are not read. A new-style arXiv id YYMM.NNNN dates a paper to the month 20YY-MM; otherwise Paper.year dates it to the year; otherwise it is undated. Dates are when the paper appeared, not when the code was written. Read 2026-11-22.

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

Counterspeech, defined as a response to mitigate online hate speech, is increasingly used as a non-censorial solution. The effectiveness of addressing hate speech involves dispelling the stereotypes, prejudices, and biases often subtly implied in brief, single-sentence statements or abuses. These expressions challenge language models, especially in seq2seq tasks, as model performance typically excels with longer contexts. Our study introduces CoARL, a novel framework enhancing counterspeech generation by modeling the pragmatic implications underlying social biases in hateful statements. The first two phases of CoARL involve sequential multi-instruction tuning, teaching the model to understand intents, reactions, and harms of offensive statements, and then learning task-specific low-rank adapter weights for generating intent-conditioned counterspeech. The final phase uses reinforcement learning to fine-tune outputs for effectiveness and nontoxicity. CoARL outperforms existing benchmarks in intent-conditioned counterspeech generation, showing an average improvement of ∼3 points in intent-conformity and ∼4 points in argument-quality metrics. Extensive human evaluation supports CoARL's efficacy in generating superior and more context-appropriate responses compared to existing systems, including prominent LLMs like ChatGPT. * * Equal contribution Hate Speech: Always quiet before the headscarves show up. Model -GPS: Women wearing headscarves, like all individuals, bring their unique cultures, ideas, and peaceful presence, enriching the tapestry of our society. Model -DialoGPT: This statement is not only inaccurate but also unfairly targets a specific group. Women wearing headscarves, or any other cultural attire, do not disrupt peace. Model -QUARC: Why would the presence of women wearing headscarves be seen as a disruption to quiet?

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2403.10088")
get_code_for_paper("2403.10088")
have("2403.10088")

The run record, dated, one paper per request, free:

curl https://syntology.ai/api/ran/2403.10088.json

A badge for a README (the split and the date, never a ratio):

[![Syntology run record](https://syntology.ai/api/ran/2403.10088.svg)](https://syntology.ai/paper/2403.10088)

Connect an agent — have() is free.