  Two new algorithms for the all-reduce operation, optimized for imbalanced
process arrival patterns (PAPs) are presented: (i) sorted linear tree (SLT),
(ii) pre-reduced ring (PRR) as well as a new way of on-line PAP detection,
including process arrival time (PAT) estimations and their distribution between
cooperating processes was introduced. The idea, pseudo-code, implementation
details, benchmark for performance evaluation and a real case example for
machine learning are provided. The results of the experiments were described
and analyzed, showing that the proposed solution has high scalability and
improved performance in comparison with the usually used ring and Rabenseifner
algorithms.
