betaFeedback

WorksComputer Science

Linear Network Coding for Sum All-Reduce over Ring Networks

Abstract

Sum All-Reduce is a fundamental collective communication primitive in which each node holds a local vector and every node must recover the componentwise sum of all vectors. In distributed training, this operation is often implemented over a ring topology using a reduce–multicast scheme. We study sum All-Reduce on directed rings via linear network coding, which includes reduce–multicast as a special case. We give a necessary and sufficient feasibility condition in terms of the coding matrices. For a ring with N≥3N\geq 3 nodes and unit-capacity links, where each node holds a vector containing KK symbols, we prove that the number of time units TT required by any feasible linear code satisfies T≥max⁡{K+N−2, ⌈2(N−1)NK⌉}T\ge\max\bigl\{K+N-2,\ \lceil\frac{2(N-1)}{N}K\rceil\bigr\}, giving a rate K/TK/T of at most N/(2(N−1))N/(2(N-1)). The bound is tight over every field. When K≥NK\ge N, an interleaved reduce–multicast code achieves the bound; when K<NK<N and KK divides NN, a reduce–partial-multicast code does. When K<NK<N and KK does not divide NN, no coordinate-wise code—in particular, no reduce–multicast code—achieves the bound, while an explicit linear network code does.

Paper details

vlnc:W_9425s49d
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC-BY-NC-ND-4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…