Dota 2 with Large Scale Deep Reinforcement Learning

Tim Salimans, Jie Tang, Jonathan Raiman, Christy Dennison, Szymon Sidor, David Farhi, Jakub Pachocki, Greg Brockman, Vicki Cheung, Jonas Schneider, Oliveira Pinto, Michael Petrov, and 15 more

arXiv · PDF · Open in the Atlas

Code that ran

We lifted 9 functions out of this paper's own repositories and ran 9 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.

Repositories linked to this paper

• bilibili/LastOrder-Dota2

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

On April 13th, 2019, OpenAI Five became the first AI system to defeat the world champions at an esports game. The game of Dota 2 presents novel challenges for AI systems such as long time horizons, imperfect information, and complex, continuous state-action spaces, all challenges which will become increasingly central to more capable AI systems. OpenAI Five leveraged existing reinforcement learning techniques, scaled to learn from batches of approximately 2 million frames every 2 seconds. We developed a distributed training system and tools for continual training which allowed us to train OpenAI Five for 10 months. By defeating the Dota 2 world champion (Team OG), OpenAI Five demonstrates that self-play reinforcement learning can achieve superhuman performance on a difficult task.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("1912.06680")
get_code_for_paper("1912.06680")
have("1912.06680")

Connect an agent — have() is free.