SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2502.15218 · NAACL · 2025

ESPnet-SpeechLM: An Open Speech Language Model Toolkit

Graham Neubig, Siddhant Arora, Shinji Watanabe, Chao-Han Huck, Yihan Wu, Yifan Peng, Xiang Yue, Jiatong Shi, William Chen, Jinchuan Tian, Samuele Cornell, Yiwen Zhao, and 4 more

arXiv · PDF · Open in the Atlas

Code that ran

We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

We present ESPnet-SpeechLM, an open toolkit designed to democratize the development of speech language models (SpeechLMs) and voice-driven agentic applications. The toolkit standardizes speech processing tasks by framing them as universal sequential modeling problems, encompassing a cohesive workflow of data preprocessing, pre-training, inference, and task evaluation. With ESPnet-SpeechLM, users can easily define task templates and configure key settings, enabling seamless and streamlined SpeechLM development. The toolkit ensures flexibility, efficiency, and scalability by offering highly configurable modules for every stage of the workflow. To illustrate its capabilities, we provide multiple use cases demonstrating how competitive SpeechLMs can be constructed with ESPnet-SpeechLM, including a 1.7B-parameter model pre-trained on both text and speech tasks, across diverse benchmarks. The toolkit and its recipes are fully transparent and reproducible at: https://github.com/ espnet/espnet/tree/speechlm.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2502.15218")
get_code_for_paper("2502.15218")
have("2502.15218")

The run record, dated, one paper per request, free:

curl https://syntology.ai/api/ran/2502.15218.json

A badge for a README (the split and the date, never a ratio):

[![Syntology run record](https://syntology.ai/api/ran/2502.15218.svg)](https://syntology.ai/paper/2502.15218)

Connect an agent — have() is free.