SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2406.10100 · 2024

SkySenseGPT: A Fine-Grained Instruction Tuning Dataset and Model for Remote Sensing Vision-Language Understanding

arXiv · PDF · Open in the Atlas

Code that ran

We lifted 6 functions out of this paper's own repositories and ran 6 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.

RepositoryRoleRan
luo-z13/skysensegpt canonical 6 of 6
FunctionStatusWhere it lives
calculate_precision_recall Ran luo-z13/skysensegpt/Eval_scripts/batch_geochat_sceneclassification.py
code served (permissive licence) · get_code("dca3ae95abe2adc0")
extract_choice_label Ran luo-z13/skysensegpt/Eval_scripts/batch_fitrsrc_single_choice_qa.py
code served (permissive licence) · get_code("1b9e414193635510")
get_chunk Ran luo-z13/skysensegpt/Eval_scripts/batch_fitrsrc_single_choice_qa.py
code served (permissive licence) · get_code("42a46570620cd9fa")
obb2poly_np_oc Ran luo-z13/skysensegpt/Eval_scripts/batch_geochat_complex_compre.py
code served (permissive licence) · get_code("86f3401508de9387")
parse_model_name Ran luo-z13/skysensegpt/Eval_scripts/langperf_imgcaption.py
code served (permissive licence) · get_code("719d4e42eca1def2")
split_list Ran luo-z13/skysensegpt/Eval_scripts/batch_fitrsrc_single_choice_qa.py
code served (permissive licence) · get_code("076c252c52cbb161")

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

Remote Sensing Large Multi-Modal Models (RSLMMs) are developing rapidly and showcase significant capabilities in remote sensing imagery (RSI) comprehension. However, due to the limitations of existing datasets, RSLMMs have shortcomings in understanding the rich semantic relations among objects in complex remote sensing scenes. To unlock RSLMMs' complex comprehension ability, we propose a large-scale instruction tuning dataset FIT-RS, containing 1,800,851 instruction samples. FIT-RS covers common interpretation tasks and innovatively introduces several complex comprehension tasks of escalating difficulty, ranging from relation reasoning to image-level scene graph generation. Based on FIT-RS, we build the FIT-RSFG benchmark. Furthermore, we establish a new benchmark to evaluate the fine-grained relation comprehension capabilities of LMMs, named FIT-RSRC. Based on combined instruction data, we propose SkySenseGPT, which achieves outstanding performance on both public datasets and FIT-RSFG, surpassing existing RSLMMs. We hope the FIT-RS dataset can enhance the relation comprehension capability of RSLMMs and provide a large-scale fine-grained data source for the remote sensing community. The dataset will be available at https://github.com/Luo-Z13/SkySenseGPT

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2406.10100")
get_code_for_paper("2406.10100")
have("2406.10100")

Connect an agent — have() is free.