Kaixuan Huang, Xiang Li, Zhihua Zhang, Wenhao Yang, Shusen Wang
We lifted 5 functions out of this paper's own repositories and ran 5 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| lx10077/fedavgpy | canonical | 3 of 3 |
| hmgxr128/mifa_code | extension | 2 of 2 |
| Function | Status | Where it lives |
|---|---|---|
| choose_two_digit | Ran | lx10077/fedavgpy/data/mnist/generate_random_niid.py code served (permissive licence) · get_code("6bd5ddf4bb11ca8b") |
| data_split | Ran | lx10077/fedavgpy/data/mnist/generate_random_niid.py code served (permissive licence) · get_code("3bc895fd7c355f2a") |
| opt_reader | Ran | hmgxr128/mifa_code/plot.py pointer only (licence: NONE) · get_code("cb1fd4ca3d3c8963") |
| smooth | Ran | lx10077/fedavgpy/plot/plot_E.py code served (permissive licence) · get_code("53730241dff8023d") |
| smooth_curve | Ran | hmgxr128/mifa_code/plot.py pointer only (licence: NONE) · get_code("9c55ec78ad29aa46") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Federated learning enables a large amount of edge computing devices to jointly learn a model without data sharing. As a leading algorithm in this setting, Federated Averaging (FedAvg) runs Stochastic Gradient Descent (SGD) in parallel on a small subset of the total devices and averages the sequences only once in a while. Despite its simplicity, it lacks theoretical guarantees under realistic settings. In this paper, we analyze the convergence of FedAvg on non-iid data and establish a convergence rate of O( 1T ) for strongly convex and smooth problems, where T is the number of SGDs. Importantly, our bound demonstrates a trade-off between communicationefficiency and convergence rate. As user devices may be disconnected from the server, we relax the assumption of full device participation to partial device participation and study different averaging schemes; low device participation rate can be achieved without severely slowing down the learning. Our results indicates that heterogeneity of data slows down the convergence, which matches empirical observations. Furthermore, we provide a necessary condition for FedAvg on non-iid data: the learning rate η must decay, even if full-gradient is used; otherwise, the solution will be Ω(η) away from the optimal.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("1907.02189")
get_code_for_paper("1907.02189")
have("1907.02189")
Connect an agent — have() is free.