SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2503.17469 · 2025

OmniLearn: A Framework for Distributed Deep Learning over Heterogeneous Clusters

Sahil Tyagi, Prateek Sharma

arXiv · PDF · Open in the Atlas

Code that ran

We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

Deep learning systems are optimized for clusters with homogeneous resources. However, heterogeneity is prevalent in computing infrastructure across edge, cloud and HPC. When training neural networks using stochastic gradient descent techniques on heterogeneous resources, performance degrades due to stragglers and stale updates. In this work, we develop an adaptive batch-scaling framework called OmniLearn to mitigate the effects of heterogeneity in distributed training. Our approach is inspired by proportional controllers to balance computation across heterogeneous servers, and works under varying resource availability. By dynamically adjusting worker mini-batches at runtime, OmniLearn reduces training time by 14-85%. We also investigate asynchronous training, where our techniques improve accuracy by up to 6.9%.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2503.17469")
get_code_for_paper("2503.17469")
have("2503.17469")

The run record, dated, one paper per request, free:

curl https://syntology.ai/api/ran/2503.17469.json

A badge for a README (the split and the date, never a ratio):

[![Syntology run record](https://syntology.ai/api/ran/2503.17469.svg)](https://syntology.ai/paper/2503.17469)

Connect an agent — have() is free.