SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2305.01528 · ACL · 2023

FIREBALL: A Dataset of Dungeons and Dragons Actual-Play with Structured Game State Information

Chris Callison-Burch, Karmanya Aggarwal, Andrew Zhu, Alexander H. Feng, Lara J. Martin

arXiv · PDF · Open in the Atlas

Code that ran

We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

Dungeons & Dragons (D&D) is a tabletop roleplaying game with complex natural language interactions between players and hidden state information. Recent work has shown that large language models (LLMs) that have access to state information can generate higher quality game turns than LLMs that use dialog history alone. However, previous work used game state information that was heuristically created and was not a true gold standard game state. We present FIREBALL, a large dataset containing nearly 25,000 unique sessions from real D&D gameplay on Discord with true game state info. We recorded game play sessions of players who used the Avrae bot, which was developed to aid people in playing D&D online, capturing language, game commands and underlying game state information. We demonstrate that FIRE-BALL can improve natural language generation (NLG) by using Avrae state information, improving both automated metrics and human judgments of quality. Additionally, we show that LLMs can generate executable Avrae commands, particularly after finetuning. RACE

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2305.01528")
get_code_for_paper("2305.01528")
have("2305.01528")

The run record, dated, one paper per request, free:

curl https://syntology.ai/api/ran/2305.01528.json

A badge for a README (the split and the date, never a ratio):

[![Syntology run record](https://syntology.ai/api/ran/2305.01528.svg)](https://syntology.ai/paper/2305.01528)

Connect an agent — have() is free.