SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 1912.05671 · 2019

Linear Mode Connectivity and the Lottery Ticket Hypothesis

Jonathan Frankle, Michael Carbin, Daniel Roy, Gintare Dziugaite

arXiv · PDF · Open in the Atlas

Code that ran

We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

We study whether a neural network optimizes to the same, linearly connected minimum under different samples of SGD noise (e.g., random data order and augmentation). We find that standard vision models become stable to SGD noise in this way early in training. From then on, the outcome of optimization is determined to a linearly connected region. We use this technique to study iterative magnitude pruning (IMP), the procedure used by work on the lottery ticket hypothesis to identify subnetworks that could have trained in isolation to full accuracy. We find that these subnetworks only reach full accuracy when they are stable to SGD noise, which either occurs at initialization for small-scale settings (MNIST) or early in training for large-scale settings (ResNet-50 and Inception-v3 on ImageNet).

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("1912.05671")
get_code_for_paper("1912.05671")
have("1912.05671")

The run record, dated, one paper per request, free:

curl https://syntology.ai/api/ran/1912.05671.json

A badge for a README (the split and the date, never a ratio):

[![Syntology run record](https://syntology.ai/api/ran/1912.05671.svg)](https://syntology.ai/paper/1912.05671)

Connect an agent — have() is free.