SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2302.05534 · NeurIPS · 2023

Robust Knowledge Transfer in Tiered RL

Niao He, Jiawei Huang

arXiv · PDF · Open in the Atlas

Code that ran

2 of 2 samples (functions and classes) lifted from this paper's linked repositories executed on a synthesized input in Syntology's jail (1 of them constructed an object rather than computing a result; 0 honoured the contract we drafted, 1 violated it and 1 has no contract verdict; 0 failures of our instrument; 0 no recorded run), measured 2026-11-22; repository links from the Papers with Code archive attributed under CC BY-SA, no affiliation; bodies inline only under a permissive licence as GitHub detects it. Not a reproduction of the paper's results.

"Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results. This record, dated, as JSON: /api/ran/2302.05534.json · badge: /api/ran/2302.05534.svg

Each repository's role is a label: a classifier's output recorded on the link, or, where no role is recorded, what the Papers with Code archive marked. The line under it names its basis. Neither says who wrote the code.

RepositoryRoleRan
jiaweihhuang/robust-tiered-rl canonical (classifier label)
classifier basis: classifier_2025
2 ran · 0 no recorded run
FunctionStatusWhere it lives
EstimatedModel Ran
ran on a synthesized input · constructed an object; no result was computed
jiaweihhuang/robust-tiered-rl/Algorithm.py
pointer only (licence: NONE) · get_code("b3fd0ca1160f27b9")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, jiaweihhuang/robust-tiered-rl when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

compute_transferable_states Ran
ran, violated the contract we drafted
jiaweihhuang/robust-tiered-rl/Algorithm.py
pointer only (licence: NONE) · get_code("621f340ede9b620e")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, jiaweihhuang/robust-tiered-rl when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

Also present in other papers' code

Of 2 samples lifted from this paper's linked repositories, 2 carry a function identity and 0 do not. Of the 2 distinct functions with an identity, 0 are also present in other papers whose code we hold (0 with at least one paper dated earlier, 0 the same month, 0 later) and 2 are only here among the 52,977 papers whose code Syntology holds (as of 2026-09-26).

Only here means: among the 52,977 papers whose code Syntology holds (51,887 with lifted samples, 1,090 through their assemblies only), as of 2026-09-26, no other paper's lifted samples or current assembly bricks contain an identical function body after normalisation.

Presence, not direction: an identical function body in two papers' repositories does not say who wrote it, which came from which, or that either paper's method rests on it. Matching is exact after normalisation, so a renamed or edited function is not matched. Dates are when papers appeared, not when code was written. Only .py files the harvester lifted are covered; notebooks are not read. A new-style arXiv id YYMM.NNNN dates a paper to the month 20YY-MM; otherwise Paper.year dates it to the year; otherwise it is undated. Dates are when the paper appeared, not when the code was written. Read 2026-11-22.

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

In this paper, we study the Tiered Reinforcement Learning setting, a parallel transfer learning framework, where the goal is to transfer knowledge from the low-tier (source) task to the high-tier (target) task to reduce the exploration risk of the latter while solving the two tasks in parallel. Unlike previous work, we do not assume the low-tier and high-tier tasks share the same dynamics or reward functions, and focus on robust knowledge transfer without prior knowledge on the task similarity. We identify a natural and necessary condition called the "Optimal Value Dominance" for our objective. Under this condition, we propose novel online learning algorithms such that, for the high-tier task, it can achieve constant regret on partial states depending on the task similarity and retain near-optimal regret when the two tasks are dissimilar, while for the low-tier task, it can keep near-optimal without making sacrifice. Moreover, we further study the setting with multiple low-tier tasks, and propose a novel transfer source selection mechanism, which can ensemble the information from all low-tier tasks and allow provable benefits on a much larger state-action space. Comparing with individual learning from scratch, transferring knowledge from other similar tasks or side information has been proven to be an effective way to reduce the exploration risk and improve sample efficiency in Reinforcement Learning (RL). Multi-Task RL (MT-RL) [29] and Transfer RL [26,18,37] are two mainstream knowledge transfer frameworks; however, both are subject to limitations when dealing with real-world scenarios. MT-RL studies the setting where a set of similar tasks are solved concurrently, and the main objective is to accelerate the learning by sharing information of all tasks together. However, in practice, in many MT-RL scenarios, the tasks are not equally important and we are more interested in the performance of certain tasks. For example, in robot learning, a few robots are more valuable and hard to fix, while the others are cheaper or just simulators. Most existing works on MT-RL treat all tasks equally and focus primarily on the reduction of the total regret of all tasks as a whole [3,8,36,11], with no guarantee of improving a particular task. In contrast, transfer RL distinguishes the priority of different tasks by categorizing them into source and target tasks and aims at transferring the knowledge from source tasks (or some side information like value predictors) to facilitate the learning of target tasks [21,27,9,10]. However, a key assumption in transfer RL is that the source task is completely solved before the learning of the target task, and this is not always practical. For example, in some sim-to-real domain, the source task simulator may require a long time to solve [5], and in some user-interaction scenarios [13], the source and target tasks refer to different user groups and they have to be served simultaneously. In these cases, it's more reasonable to solve the source and target tasks in parallel and transfer the information immediately once available. Recently, [13] proposed a new "parallel knowledge transfer" framework, called Tiered RL, which is promising to fill the gap. Tiered RL considers the case when a source task M Lo and a target task 37th Conference on Neural Information Processing Systems (NeurIPS 2023).

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2302.05534")
get_code_for_paper("2302.05534")
have("2302.05534")

The run record, dated, one paper per request, free:

curl https://syntology.ai/api/ran/2302.05534.json

A badge for a README (the split and the date, never a ratio):

[![Syntology run record](https://syntology.ai/api/ran/2302.05534.svg)](https://syntology.ai/paper/2302.05534)

Connect an agent — have() is free.