SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2306.02418 · ICML · 2023

ContraBAR: Contrastive Bayes-Adaptive Deep RL

Aviv Tamar, Era Choshen

arXiv · PDF · Open in the Atlas

Code that ran

We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

In meta reinforcement learning (meta RL), an agent seeks a Bayes-optimal policy -the optimal policy when facing an unknown task that is sampled from some known task distribution. Previous approaches tackled this problem by inferring a belief over task parameters, using variational inference methods. Motivated by recent successes of contrastive learning approaches in RL, such as contrastive predictive coding (CPC), we investigate whether contrastive methods can be used for learning Bayes-optimal behavior. We begin by proving that representations learned by CPC are indeed sufficient for Bayes optimality. Based on this observation, we propose a simple meta RL algorithm that uses CPC in lieu of variational belief inference. Our method, ContraBAR, achieves comparable performance to state-of-the-art in domains with state-based observation and circumvents the computational toll of future observation reconstruction, enabling learning in domains with image-based observations. It can also be combined with image augmentations for domain randomization and used seamlessly in both online and offline meta RL settings.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2306.02418")
get_code_for_paper("2306.02418")
have("2306.02418")

The run record, dated, one paper per request, free:

curl https://syntology.ai/api/ran/2306.02418.json

A badge for a README (the split and the date, never a ratio):

[![Syntology run record](https://syntology.ai/api/ran/2306.02418.svg)](https://syntology.ai/paper/2306.02418)

Connect an agent — have() is free.