SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2012.10333 · ICML · 2021

Learning from History for Byzantine Robust Optimization

Martin Jaggi, Sai Praneeth Karimireddy, Lie He

arXiv · PDF · Open in the Atlas

Code that ran

We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

Byzantine robustness has received significant attention recently given its importance for distributed and federated learning. In spite of this, we identify severe flaws in existing algorithms even when the data across the participants is identically distributed. First, we show realistic examples where current state of the art robust aggregation rules fail to converge even in the absence of any Byzantine attackers. Secondly, we prove that even if the aggregation rules may succeed in limiting the influence of the attackers in a single round, the attackers can couple their attacks across time eventually leading to divergence. To address these issues, we present two surprisingly simple strategies: a new robust iterative clipping procedure, and incorporating worker momentum to overcome time-coupled attacks. This is the first provably robust method for the standard stochastic optimization setting. Our code is open sourced at this link 2 .

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2012.10333")
get_code_for_paper("2012.10333")
have("2012.10333")

The run record, dated, one paper per request, free:

curl https://syntology.ai/api/ran/2012.10333.json

A badge for a README (the split and the date, never a ratio):

[![Syntology run record](https://syntology.ai/api/ran/2012.10333.svg)](https://syntology.ai/paper/2012.10333)

Connect an agent — have() is free.