SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2101.03682 · ICCV · 2021

MAAS: Multi-modal Assignation for Active Speaker Detection

Bernard Ghanem, Ali Thabet, Fabian Caba Heilbron, Juan León Alcázar

arXiv · PDF · Open in the Atlas

Code that ran

We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

Active speaker detection requires a mindful integration of multi-modal cues. Current methods focus on modeling and fusing short-term audiovisual features for individual speakers, often at frame level. We present a novel approach to active speaker detection that directly addresses the multimodal nature of the problem and provides a straightforward strategy, where independent visual features (speakers) in the scene are assigned to a previously detected speech event. Our experiments show that a small graph data structure built from local information can approximate an instantaneous audio-visual assignment problem. Moreover, the temporal extension of this initial graph achieves a new state-of-the-art performance on the AVA-ActiveSpeaker dataset with a mAP of 88.8%. a) Audiovisual Feature Encoding b) Multi-modal Graph Network d) Temporal Assignation

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2101.03682")
get_code_for_paper("2101.03682")
have("2101.03682")

The run record, dated, one paper per request, free:

curl https://syntology.ai/api/ran/2101.03682.json

A badge for a README (the split and the date, never a ratio):

[![Syntology run record](https://syntology.ai/api/ran/2101.03682.svg)](https://syntology.ai/paper/2101.03682)

Connect an agent — have() is free.