We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. We have no repository linked to it either. That is a statement about our graph, not about the paper.
In this paper we present a text-conditioned video resampler (TCR) module that uses a pre-trained and frozen visual encoder and large language model (LLM) to process long video sequences for a task. TCR localises relevant visual features from the video given a text condition and provides them to a LLM to generate a text response. Due to its lightweight design and use of cross-attention, TCR can process more than 100 frames at a time with plain attention and without optimised implementations. We make the following contributions: (i) we design a transformer-based sampling architecture that can process long videos conditioned on a task, together with a training method that enables it to bridge pre-trained visual and language models; (ii) we identify tasks that could benefit from longer video perception; and (iii) we empirically validate its efficacy on a wide variety of evaluation tasks including NextQA, EgoSchema, and the EGO4D-LTA challenge.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2312.11897")
get_code_for_paper("2312.11897")
have("2312.11897")
The run record, dated, one paper per request, free:
curl https://syntology.ai/api/ran/2312.11897.json
A badge for a README (the split and the date, never a ratio):
[](https://syntology.ai/paper/2312.11897)
Connect an agent — have() is free.