SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2209.10320 · NeurIPS · 2022

Continual VQA for Disaster Response Systems

V Manushree, Sahil Khose, Aditya Kane

arXiv · PDF · Open in the Atlas

Code that ran

We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

Visual Question Answering (VQA) is a multi-modal task that involves answering questions from an input image, semantically understanding the contents of the image and answering it in natural language. Using VQA for disaster management is an important line of research due to the scope of problems that are answered by the VQA system. However, the main challenge is the delay caused by the generation of labels in the assessment of the affected areas. To tackle this, we deployed pre-trained CLIP model, which is trained on visual-image pairs. however, we empirically see that the model has poor zero-shot performance. Thus, we instead use pre-trained embeddings of text and image from this model for our supervised training and surpass previous state-of-the-art results on the FloodNet dataset. We expand this to a continual setting, which is a more real-life scenario. We tackle the problem of catastrophic forgetting using various experience replay methods. 2 * Indicates equal contribution 2 Our training runs are available at: https://wandb.ai/compyle/continual_vqa_final Preprint. Under review.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2209.10320")
get_code_for_paper("2209.10320")
have("2209.10320")

The run record, dated, one paper per request, free:

curl https://syntology.ai/api/ran/2209.10320.json

A badge for a README (the split and the date, never a ratio):

[![Syntology run record](https://syntology.ai/api/ran/2209.10320.svg)](https://syntology.ai/paper/2209.10320)

Connect an agent — have() is free.