Lina Yao, Xiaojun Chang, Heng Wang, Linjie Yang, Mingfei Han, Reler Lab
We lifted 7 functions out of this paper's own repositories and ran 6 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| bytedance/Shot2Story | canonical | 6 of 7 |
| Function | Status | Where it lives |
|---|---|---|
| generate_caption | Ran | bytedance/Shot2Story/code/app/caption.py pointer only (licence: NONE) · get_code("56f02812d66a0d11") |
| get_concat_v | Ran | bytedance/Shot2Story/code/app/dataset_browser.py pointer only (licence: NONE) · get_code("92f2e16ee0d3a24d") |
| read_img | Ran | bytedance/Shot2Story/code/app/calculate_coco_features.py pointer only (licence: NONE) · get_code("13a43a7d815937e6") |
| resize_img | Ran | bytedance/Shot2Story/code/app/utils.py pointer only (licence: NONE) · get_code("926bb38f979a61ef") |
| resize_img_w | Ran | bytedance/Shot2Story/code/app/dataset_browser.py pointer only (licence: NONE) · get_code("f809f2ecf34c1c99") |
| sample_dataset | Ran | bytedance/Shot2Story/code/app/dataset_browser.py pointer only (licence: NONE) · get_code("3d4bbb5ca53fd24a") |
| compute_gradcam_batch | Not yet run | bytedance/Shot2Story/code/app/multimodal_search.py pointer only (licence: NONE) · get_code("3e12ea8c09674aa8") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
A short clip of video may contain progression of multiple events and an interesting story line. A human need to capture both the event in every shot and associate them together to understand the story behind it. In this work, we present a new multishot video understanding benchmark Shot2Story with detailed shot-level captions, comprehensive video summaries and question-answering pairs. To facilitate better semantic understanding of videos, we provide captions for both visual signals and human narrations. We design several distinct tasks including single-shot video captioning, multi-shot video summarization, and multi-shot video question answering. Preliminary experiments show some challenges to generate a long and comprehensive video summary for multi-shot videos. Nevertheless, the generated imperfect summaries can already achieve competitive performance on existing video understanding tasks such as video question-answering, promoting an underexplored setting of video understanding with detailed summaries.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2312.10300")
get_code_for_paper("2312.10300")
have("2312.10300")
Connect an agent — have() is free.