We lifted 4 functions out of this paper's own repositories and ran 2 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| coldmanck/VidHOI | canonical | 2 of 4 |
| Function | Status | Where it lives |
|---|---|---|
| construct_optimizer | Ran | coldmanck/VidHOI/slowfast/models/optimizer.py pointer only (licence: Apache-2.0) · get_code("75c62c582a31b362") |
| get_loss_func | Ran | coldmanck/VidHOI/slowfast/models/losses.py pointer only (licence: Apache-2.0) · get_code("c90b5f606f54d4f7") |
| get_trans_func | Not yet run | coldmanck/VidHOI/slowfast/models/resnet_helper.py pointer only (licence: Apache-2.0) · get_code("4d6991202a8d6097") |
| interaction_inference_single_image | Not yet run | coldmanck/VidHOI/slowfast/models/fast_rcnn.py pointer only (licence: Apache-2.0) · get_code("e702f7e23a23d4d5") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Detecting human-object interactions (HOI) is an important step toward a comprehensive visual understanding of machines. While detecting non-temporal HOIs (e.g., sitting on a chair) from static images is feasible, it is unlikely even for humans to guess temporal-related HOIs (e.g., opening/closing a door) from a single video frame, where the neighboring frames play an essential role. However, conventional HOI methods operating on only static images have been used to predict temporal-related interactions, which is essentially guessing without temporal contexts and may lead to sub-optimal performance. In this paper, we bridge this gap by detecting video-based HOIs with explicit temporal information. We first show that a naive temporal-aware variant of a common action detection baseline does not work on video-based HOIs due to a feature-inconsistency issue. We then propose a simple yet effective architecture named Spatial-Temporal HOI Detection (ST-HOI) utilizing temporal information such as human and object trajectories, correctly-localized visual features, and spatial-temporal masking pose features. We construct a new video HOI benchmark dubbed VidHOI where our proposed approach serves as a solid baseline.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2105.11731")
get_code_for_paper("2105.11731")
have("2105.11731")
Connect an agent — have() is free.