SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2108.10668 · ICCV · 2021

Temporal Knowledge Consistency for Unsupervised Visual Representation Learning

Chi Zhang, Ye Yuan, Yuanjiang Wang, Weixin Feng, Lihua Ma

arXiv · PDF · Open in the Atlas

Code that ran

We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

The instance discrimination paradigm has become dominant in unsupervised learning. It always adopts a teacherstudent framework, in which the teacher provides embedded knowledge as a supervision signal for the student. The student learns meaningful representations by enforcing instance spatial consistency with the views from the teacher. However, the outputs of the teacher can vary dramatically on the same instance during different training stages, introducing unexpected noise and leading to catastrophic forgetting caused by inconsistent objectives. In this paper, we first integrate instance temporal consistency into current instance discrimination paradigms, and propose a novel and strong algorithm named Temporal Knowledge Consistency (TKC). Specifically, our TKC dynamically ensembles the knowledge of temporal teachers and adaptively selects useful information according to its importance to learning instance temporal consistency. Experimental result shows that TKC can learn better visual representations on both ResNet and AlexNet on linear evaluation protocol while transfer well to downstream tasks. All experiments suggest the good effectiveness and generalization of our method. Code will be made available.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2108.10668")
get_code_for_paper("2108.10668")
have("2108.10668")

The run record, dated, one paper per request, free:

curl https://syntology.ai/api/ran/2108.10668.json

A badge for a README (the split and the date, never a ratio):

[![Syntology run record](https://syntology.ai/api/ran/2108.10668.svg)](https://syntology.ai/paper/2108.10668)

Connect an agent — have() is free.