SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2004.08154 · 2020

Detailed 2D-3D Joint Representation for Human-Object Interaction

arXiv · PDF · Open in the Atlas

Code that ran

We lifted 12 functions out of this paper's own repositories and ran 8 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.

RepositoryRoleRan
DirtyHarryLYL/DJ-RN canonical 8 of 12
FunctionStatusWhere it lives
Augmented_box Ran DirtyHarryLYL/DJ-RN/lib/ult/ult.py
code served (permissive licence) · get_code("4c4b9fe1c3921060")
bb_IOU Ran DirtyHarryLYL/DJ-RN/lib/ult/ult.py
code served (permissive licence) · get_code("a945b93e1e20006e")
bbox_trans Ran DirtyHarryLYL/DJ-RN/lib/ult/ult.py
code served (permissive licence) · get_code("e7e5af90a9931839")
calc_hit Ran DirtyHarryLYL/DJ-RN/-Results/Evaluate_ambiguous.py
code served (permissive licence) · get_code("77951edeecdb7708")
calc_hit Ran DirtyHarryLYL/DJ-RN/-Results/HICO_DET_utils.py
code served (permissive licence) · get_code("be7025a18e9a78f8")
getSigmoid Ran DirtyHarryLYL/DJ-RN/-Results/Evaluate_ambiguous.py
code served (permissive licence) · get_code("9c36467d91f24d5d")
iou Ran DirtyHarryLYL/DJ-RN/-Results/Evaluate_ambiguous.py
code served (permissive licence) · get_code("434bd4b19837432c")
iou Ran DirtyHarryLYL/DJ-RN/-Results/HICO_DET_utils.py
code served (permissive licence) · get_code("082c2f358be049b5")
draw_bounding_boxes Not yet run DirtyHarryLYL/DJ-RN/lib/ult/visualization.py
code served (permissive licence) · get_code("c83a0fe665671edc")
draw_bounding_boxes_HOI Not yet run DirtyHarryLYL/DJ-RN/lib/ult/visualization.py
code served (permissive licence) · get_code("6e77405876116b07")
draw_bounding_boxes_PVP Not yet run DirtyHarryLYL/DJ-RN/lib/ult/visualization.py
code served (permissive licence) · get_code("b5831b48920061b2")
placeholder_inputs Not yet run DirtyHarryLYL/DJ-RN/Feature_extraction/pointnet_hico.py
code served (permissive licence) · get_code("f4d4327d825f56f5")

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

Human-Object Interaction (HOI) detection lies at the core of action understanding. Besides 2D information such as human/object appearance and locations, 3D pose is also usually utilized in HOI learning since its view-independence. However, rough 3D body joints just carry sparse body information and are not sufficient to understand complex interactions. Thus, we need detailed 3D body shape to go further. Meanwhile, the interacted object in 3D is also not fully studied in HOI learning. In light of these, we propose a detailed 2D-3D joint representation learning method. First, we utilize the single-view human body capture method to obtain detailed 3D body, face and hand shapes. Next, we estimate the 3D object location and size with reference to the 2D human-object spatial configuration and object category priors. Finally, a joint learning framework and cross-modal consistency tasks are proposed to learn the joint HOI representation. To better evaluate the 2D ambiguity processing capacity of models, we propose a new benchmark named Ambiguous-HOI consisting of hard ambiguous images. Extensive experiments in large-scale HOI benchmark and Ambiguous-HOI show impressive effectiveness of our method. Code and data are available at https://github.com/DirtyHarryLYL/DJ-RN.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2004.08154")
get_code_for_paper("2004.08154")
have("2004.08154")

Connect an agent — have() is free.