Kate Saenko, Mit-Ibm Watson, A Lab, Alex Bronstein, Prasanna Sattigeri, Rameswar Panda, Rogerio Feris, Shimon Ullman, Hilde Kuehne, Chun-Fu Chen, Leonid Karlinsky, Raja Giryes, and 7 more
We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Nowadays, there is an abundance of data involving images and surrounding free-form text weakly corresponding to those images. Weakly Supervised phrase-Grounding (WSG) deals with the task of using this data to learn to localize (or to ground) arbitrary text phrases in images without any additional annotations. However, most recent SotA methods for WSG assume an existence of a pre-trained object detector, relying on it to produce the ROIs for localization. In this work, we focus on the task of Detector-Free WSG (DF-WSG) to solve WSG without relying on a pretrained detector. The key idea behind our proposed Grounding by Separation (GbS) method is synthesizing 'text to image-regions' associations by random alpha-blending of arbitrary image pairs and using the corresponding texts of the pair as conditions to recover the alpha map from the blended image via a segmentation network. At test time, this allows using the query phrase as a condition for a nonblended query image, thus interpreting the test image as a composition of a region corresponding to the phrase and the complement region. Our GbS shows an 8.5% accuracy improvement over previous DF-WSG SotA, for a range of benchmarks including Flickr30K, Visual Genome, and ReferIt, as well as a complementary improvement (above 7%) over the detector-based approaches for WSG.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2104.09829")
get_code_for_paper("2104.09829")
have("2104.09829")
The run record, dated, one paper per request, free:
curl https://syntology.ai/api/ran/2104.09829.json
A badge for a README (the split and the date, never a ratio):
[](https://syntology.ai/paper/2104.09829)
Connect an agent — have() is free.