We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. We have no repository linked to it either. That is a statement about our graph, not about the paper.
As the size and complexity of models and datasets grow, so does the need for communication-efficient variants of stochastic gradient descent that can be deployed to perform parallel model training. One popular communication-compression method for data-parallel SGD is QSGD (Alistarh et al., 2017), which quantizes and encodes gradients to reduce communication costs. The baseline variant of QSGD provides strong theoretical guarantees, however, for practical purposes, the authors proposed a heuristic variant which we call QSGDinf, which demonstrated impressive empirical gains for distributed training of large neural networks. In this paper, we build on this work to propose a new gradient quantization scheme, and show that it has both stronger theoretical guarantees than QSGD, and matches and exceeds the empirical performance of the QSGDinf heuristic and of other compression methods.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2104.13818")
get_code_for_paper("2104.13818")
have("2104.13818")
The run record, dated, one paper per request, free:
curl https://syntology.ai/api/ran/2104.13818.json
A badge for a README (the split and the date, never a ratio):
[](https://syntology.ai/paper/2104.13818)
Connect an agent — have() is free.