SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2206.06291 · CVPR · 2022

Exploring Structure-aware Transformer over Interaction Proposals for Human-Object Interaction Detection *

Yingwei Pan, Ting Yao, Tao Mei, Rui Huang, Yong Zhang, Chang-Wen Chen

arXiv · PDF · Open in the Atlas

Code that ran

We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

Recent high-performing Human-Object Interaction (HOI) detection techniques have been highly influenced by Transformer-based object detector (i.e., DETR). Nevertheless, most of them directly map parametric interaction queries into a set of HOI predictions through vanilla Transformer in a one-stage manner. This leaves rich interor intra-interaction structure under-exploited. In this work, we design a novel Transformer-style HOI detector, i.e., Structure-aware Transformer over Interaction Proposals (STIP), for HOI detection. Such design decomposes the process of HOI set prediction into two subsequent phases, i.e., an interaction proposal generation is first performed, and then followed by transforming the nonparametric interaction proposals into HOI predictions via a structure-aware Transformer. The structure-aware Transformer upgrades vanilla Transformer by encoding additionally the holistically semantic structure among interaction proposals as well as the locally spatial structure of human/object within each interaction proposal, so as to strengthen HOI predictions. Extensive experiments conducted on V-COCO and HICO-DET benchmarks have demonstrated the effectiveness of STIP, and superior results are reported when comparing with the stateof-the-art HOI detectors. Source code is available at https://github.com/zyong812/STIP. * This work was performed at JD Explore Academy. Vanilla Transformer Detection Interaction Proposal Network (a) Previous Transformer-style HOI detectors (b) Structure-aware Transformer over Interaction Proposals (STIP) CNN Parametric interaction queries Structure-aware Transformer CNN <human, glove, wear> <human, bat, hold> No interaction <human, glove, wear> <human, bat, hold> No interaction human-bat human-bat human-bat human-glove Intra-interaction spatial structure Inter-interaction semantic structure

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2206.06291")
get_code_for_paper("2206.06291")
have("2206.06291")

The run record, dated, one paper per request, free:

curl https://syntology.ai/api/ran/2206.06291.json

A badge for a README (the split and the date, never a ratio):

[![Syntology run record](https://syntology.ai/api/ran/2206.06291.svg)](https://syntology.ai/paper/2206.06291)

Connect an agent — have() is free.