SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2202.00769 · NeurIPS · 2024

Distributional Reinforcement Learning with Regularized Wasserstein Loss

Wulong Liu, Ke Sun, Yingnan Zhao, Bei Jiang, Linglong Kong

arXiv · PDF · Open in the Atlas

Code that ran

We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

The empirical success of distributional reinforcement learning (RL) highly relies on the choice of distribution divergence equipped with an appropriate distribution representation. In this paper, we propose Sinkhorn distributional RL (Sinkhorn-DRL), which leverages Sinkhorn divergence-a regularized Wasserstein loss-to minimize the difference between current and target Bellman return distributions. Theoretically, we prove the contraction properties of SinkhornDRL, aligning with the interpolation nature of Sinkhorn divergence between Wasserstein distance and Maximum Mean Discrepancy (MMD). The introduced SinkhornDRL enriches the family of distributional RL algorithms, contributing to interpreting the algorithm behaviors compared with existing approaches by our investigation into their relationships. Empirically, we show that SinkhornDRL consistently outperforms or matches existing algorithms on the Atari games suite and particularly stands out in the multi-dimensional reward setting. Code is available in https://github.com/datake/SinkhornDistRL..

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2202.00769")
get_code_for_paper("2202.00769")
have("2202.00769")

The run record, dated, one paper per request, free:

curl https://syntology.ai/api/ran/2202.00769.json

A badge for a README (the split and the date, never a ratio):

[![Syntology run record](https://syntology.ai/api/ran/2202.00769.svg)](https://syntology.ai/paper/2202.00769)

Connect an agent — have() is free.