SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2110.14754 · NeurIPS · 2021

Confidence-Aware Imitation Learning from Demonstrations with Varying Optimality

Dorsa Sadigh, Zhangjie Cao, Songyuan Zhang, Yanan Sui

arXiv · PDF · Open in the Atlas

Code that ran

We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

Most existing imitation learning approaches assume the demonstrations are drawn from experts who are optimal, but relaxing this assumption enables us to use a wider range of data. Standard imitation learning may learn a suboptimal policy from demonstrations with varying optimality. Prior works use confidence scores or rankings to capture beneficial information from demonstrations with varying optimality, but they suffer from many limitations, e.g., manually annotated confidence scores or high average optimality of demonstrations. In this paper, we propose a general framework to learn from demonstrations with varying optimality that jointly learns the confidence score and a well-performing policy. Our approach, Confidence-Aware Imitation Learning (CAIL) learns a well-performing policy from confidence-reweighted demonstrations, while using an outer loss to track the performance of our model and to learn the confidence. We provide theoretical guarantees on the convergence of CAIL and evaluate its performance in both simulated and real robot experiments. Our results show that CAIL significantly outperforms other imitation learning methods from demonstrations with varying optimality. We further show that even without access to any optimal demonstrations, CAIL can still learn a successful policy, and outperforms prior work. * Equal contribution. 35th Conference on Neural Information Processing Systems (NeurIPS 2021).

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2110.14754")
get_code_for_paper("2110.14754")
have("2110.14754")

The run record, dated, one paper per request, free:

curl https://syntology.ai/api/ran/2110.14754.json

A badge for a README (the split and the date, never a ratio):

[![Syntology run record](https://syntology.ai/api/ran/2110.14754.svg)](https://syntology.ai/paper/2110.14754)

Connect an agent — have() is free.