Bernard Ghanem, Ali Thabet, Fabian Caba Heilbron, Juan León Alcázar
We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Active speaker detection requires a mindful integration of multi-modal cues. Current methods focus on modeling and fusing short-term audiovisual features for individual speakers, often at frame level. We present a novel approach to active speaker detection that directly addresses the multimodal nature of the problem and provides a straightforward strategy, where independent visual features (speakers) in the scene are assigned to a previously detected speech event. Our experiments show that a small graph data structure built from local information can approximate an instantaneous audio-visual assignment problem. Moreover, the temporal extension of this initial graph achieves a new state-of-the-art performance on the AVA-ActiveSpeaker dataset with a mAP of 88.8%. a) Audiovisual Feature Encoding b) Multi-modal Graph Network d) Temporal Assignation
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2101.03682")
get_code_for_paper("2101.03682")
have("2101.03682")
The run record, dated, one paper per request, free:
curl https://syntology.ai/api/ran/2101.03682.json
A badge for a README (the split and the date, never a ratio):
[](https://syntology.ai/paper/2101.03682)
Connect an agent — have() is free.