Boqing Gong, Ming-Hsuan Yang, Yin Cui, Long Zhao, Ting Liu, Hartwig Adam, Liangzhe Yuan, Florian Schroff
We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
This work focuses on training a single visual relationship detector predicting over the union of label spaces from multiple datasets. Merging labels spanning different datasets could be challenging due to inconsistent taxonomies. The issue is exacerbated in visual relationship detection when second-order visual semantics are introduced between pairs of objects. To address this challenge, we propose UniVRD, a novel bottom-up method for Unified Visual Relationship Detection by leveraging vision and language models (VLMs). VLMs provide well-aligned image and text embeddings, where similar relationships are optimized to be close to each other for semantic unification. Our bottom-up design enables the model to enjoy the benefit of training with both object detection and visual relationship datasets. Empirical results on both human-object interaction detection and scene-graph generation demonstrate the competitive performance of our model. UniVRD achieves 38.07 mAP on HICO-DET, outperforming the current best bottom-up HOI detector by 14.26 mAP. More importantly, we show that our unified detector performs as well as dataset-specific models in mAP, and achieves further improvements when we scale up the model. Our code will be made publicly available on GitHub 1 .
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2303.08998")
get_code_for_paper("2303.08998")
have("2303.08998")
The run record, dated, one paper per request, free:
curl https://syntology.ai/api/ran/2303.08998.json
A badge for a README (the split and the date, never a ratio):
[](https://syntology.ai/paper/2303.08998)
Connect an agent — have() is free.