Subarna Tripathi, Kien Nguyen, Truong Nguyen, Tanaya Guha
We lifted 10 functions out of this paper's own repositories and ran 4 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| kien085/sg2caps | — | 2 of 8 |
| copy not recorded | — | 2 of 2 |
| Function | Status | Where it lives |
|---|---|---|
| imge2sene_fc | Ran | kien085/sg2caps/models/AttModel_mem4.py pointer only (licence: NONE) · get_code("5f7c53446459d28b") |
| pack_wrapper | Ran | kien085/sg2caps/models/AttModel_mem4.py pointer only (licence: NONE) · get_code("537642f95d45af38") |
| pad_unsort_packed_sequence | Ran | this paper's copy was not recorded; identical code first harvested from kangkang59812/GraphCaption pointer only · get_code("bfac58a04b6835f5") |
| sort_pack_padded_sequence | Ran | this paper's copy was not recorded; identical code first harvested from kangkang59812/GraphCaption pointer only · get_code("e56d2f9cbd9cb8b1") |
| AttModel_mem4 | Not yet run | kien085/sg2caps/models/AttModel_mem4.py pointer only (licence: NONE) · get_code("95ca82a8bbb7bb07") |
| Attention | Not yet run | kien085/sg2caps/models/AttModel_mem4.py pointer only (licence: NONE) · get_code("262edaef058c9fc5") |
| CaptionModel | Not yet run | kien085/sg2caps/models/AttModel_mem4.py pointer only (licence: NONE) · get_code("7b9f3c88ca494f33") |
| Memory_cell2 | Not yet run | kien085/sg2caps/models/AttModel_mem4.py pointer only (licence: NONE) · get_code("fd479c534bc6676d") |
| TopDownCore_mem | Not yet run | kien085/sg2caps/models/AttModel_mem4.py pointer only (licence: NONE) · get_code("838295963efb02ce") |
| TopDownCore_rela | Not yet run | kien085/sg2caps/models/AttModel_mem4.py pointer only (licence: NONE) · get_code("458814ff72697a97") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
The mainstream image captioning models rely on Convolutional Neural Network (CNN) image features to generate captions via recurrent models. Recently, image scene graphs have been used to augment captioning models so as to leverage their structural semantics, such as object entities, relationships and attributes. Several studies have noted that the naive use of scene graphs from a blackbox scene graph generator harms image captioning performance and that scene graph-based captioning models have to incur the overhead of explicit use of image features to generate decent captions. Addressing these challenges, we propose SG2Caps, a framework that utilizes only the scene graph labels for competitive image captioning performance. The basic idea is to close the semantic gap between the two scene graphs -one derived from the input image and the other from its caption. In order to achieve this, we leverage the spatial location of objects and the Human-Object-Interaction (HOI) labels as an additional HOI graph. SG2Caps outperforms existing scene graph-only captioning models by a large margin, indicating scene graphs as a promising representation for image captioning. Direct utilization of scene graph labels avoids expensive graph convolutions over high-dimensional CNN features resulting in 49% fewer trainable parameters. Our code is available at: https://github.com/ Kien085/SG2Caps
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2102.04990")
get_code_for_paper("2102.04990")
have("2102.04990")
Connect an agent — have() is free.