SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2407.16417 · 2024

On the Utility of Speech and Audio Foundation Models for Marmoset Call Analysis

arXiv · PDF · Open in the Atlas

Code that ran

We lifted 5 functions out of this paper's own repositories and ran 2 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.

RepositoryRoleRan
idiap/speech-utility-bioacoustics canonical 2 of 5
FunctionStatusWhere it lives
collate Ran idiap/speech-utility-bioacoustics/src/extract_byol.py
pointer only (licence: GPL-3.0) · get_code("8dcb423585156d31")
unpad Ran idiap/speech-utility-bioacoustics/src/extract_byol.py
pointer only (licence: GPL-3.0) · get_code("213879b8eb3bebd4")
collate_fn Not yet run idiap/speech-utility-bioacoustics/src/utils/loader.py
pointer only (licence: GPL-3.0) · get_code("62552bdaa51f0964")
manual_reflect_pad Not yet run idiap/speech-utility-bioacoustics/src/utils/loader.py
pointer only (licence: GPL-3.0) · get_code("05f2bdc0c7b2e628")
pad_sequence Not yet run idiap/speech-utility-bioacoustics/src/utils/loader.py
pointer only (licence: GPL-3.0) · get_code("8ae639e28a78279a")

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

Marmoset monkeys encode vital information in their calls and serve as a surrogate model for neuro-biologists to understand the evolutionary origins of human vocal communication. Traditionally analyzed with signal processing-based features, recent approaches have utilized self-supervised models pre-trained on human speech for feature extraction, capitalizing on their ability to learn a signal's intrinsic structure independently of its acoustic domain. However, the utility of such foundation models remains unclear for marmoset call analysis in terms of multi-class classification, bandwidth, and pre-training domain. This study assesses feature representations derived from speech and general audio domains, across pre-training bandwidths of 4, 8, and 16 kHz for marmoset call-type and caller classification tasks. Results show that models with higher bandwidth improve performance, and pre-training on speech or general audio yields comparable results, improving over a spectral baseline.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2407.16417")
get_code_for_paper("2407.16417")
have("2407.16417")

Connect an agent — have() is free.