MoSiC: Optimal-Transport Motion Trajectory for Dense Self-Supervised Learning

Cees Snoek, Efstratios Gavves, Yuki Asano, Valeo Ai, Shashanka Venkataramanan, Mohammadreza Salehi, Ioana Simion, Vis Lab, Uva

arXiv · PDF · Open in the Atlas

Code that ran

We lifted 8 functions out of this paper's own repositories and ran 7 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.

Repositories linked to this paper

• smsd75/mosic — marked official in Papers with Code; named in the paper
• SMSD75/MoSiC

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

Dense self-supervised learning has shown great promise for learning pixel-and patch-level representations, but extending it to videos remains challenging due to the complexity of motion dynamics. Existing approaches struggle as they rely on static augmentations that fail under object deformations, occlusions, and camera movement, leading to inconsistent feature learning over time. We propose a motion-guided self-supervised learning framework that clusters dense point tracks to learn spatiotemporally consistent representations. By leveraging an off-the-shelf point tracker, we extract long-range motion trajectories and optimize feature clustering through a momentum-encoder-based optimal transport mechanism. To ensure temporal coherence, we propagate cluster assignments along tracked points, enforcing feature consistency across views despite viewpoint changes. Integrating motion as an implicit supervisory signal, our method learns representations that generalize across frames, improving robustness in dynamic scenes and challenging occlusion scenarios. By initializing from strong image-pretrained models and leveraging video data for training, we improve state-of-the-art by 1% to 6% on six image and video datasets and four evaluation benchmarks. The implementation is publicly available at our GitHub repository: github.com/SMSD75/MoSiC

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2506.08694")
get_code_for_paper("2506.08694")
have("2506.08694")

Connect an agent — have() is free.