SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2005.10243 · NeurIPS · 2020

What Makes for Good Views for Contrastive Learning?

Ben Poole, Phillip Isola, Yonglong Tian, Dilip Krishnan, Cordelia Schmid, Chen Sun

arXiv · PDF · Open in the Atlas

Code that ran

We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

Contrastive learning between multiple views of the data has recently achieved state of the art performance in the field of self-supervised representation learning. Despite its success, the influence of different view choices has been less studied. In this paper, we use theoretical and empirical analysis to better understand the importance of view selection, and argue that we should reduce the mutual information (MI) between views while keeping task-relevant information intact. To verify this hypothesis, we devise unsupervised and semi-supervised frameworks that learn effective views by aiming to reduce their MI. We also consider data augmentation as a way to reduce MI, and show that increasing data augmentation indeed leads to decreasing MI and improves downstream classification accuracy. As a byproduct, we achieve a new state-of-the-art accuracy on unsupervised pre-training for ImageNet classification (73% top-1 linear readout with a ResNet-50) 1 .

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2005.10243")
get_code_for_paper("2005.10243")
have("2005.10243")

The run record, dated, one paper per request, free:

curl https://syntology.ai/api/ran/2005.10243.json

A badge for a README (the split and the date, never a ratio):

[![Syntology run record](https://syntology.ai/api/ran/2005.10243.svg)](https://syntology.ai/paper/2005.10243)

Connect an agent — have() is free.