Roy Schwartz, Jack Hessel, Ludwig Schmidt, Yonatan Bitton, Nitzan Bitton-Guetta, Yuval Elovici, Gabriel Stanovsky
We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. If it links a repository, it is listed below.
Weird, unusual, and uncanny images pique the curiosity of observers because they challenge commonsense. For example, an image released during the 2022 world cup depicts the famous soccer stars Lionel Messi and Cristiano Ronaldo playing chess, which playfully violates our expectation that their competition should occur on the football field. 1 Humans can easily recognize and interpret these unconventional images, but can AI models do the same? We introduce WHOOPS!, a new dataset and benchmark for visual commonsense. The dataset is comprised of purposefully commonsense-defying images created by designers using publicly-available image generation tools like Midjourney. We consider several tasks posed over the dataset. In addition to image captioning, cross-modal matching, and visual question answering, we introduce a difficult explanation generation task, where models must identify and explain why a given image is unusual. Our results show that state-of-the-art models such as GPT3 and BLIP2 still lag behind human performance on WHOOPS!. We hope our dataset will inspire the development of AI models with stronger visual commonsense reasoning abilities. 2 Pandas live in the China bamboo forests, subsist almost entirely on bamboo, and do not hunt salmon fish like the grizzly bears. A panda bear is catching salmon fish up in the river stream. Model VQA Model Q: What is catching salmon in a stream? Q: What does a panda bear try to catch? A panda A salmon
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2303.07274")
get_code_for_paper("2303.07274")
have("2303.07274")
Connect an agent — have() is free.