Eager Updates For Overlapped Communication and Computation in DiLoCo
Paper
•
2502.12996
•
Published
•
7
Arthur Douillard
ArthurDouillard
AI & ML interests
Continual Learning, Computer Vision, Transformers
Organizations
None yet
authored
a
paper
8 months ago
authored
5
papers
over 1 year ago
DiLoCo: Distributed Low-Communication Training of Language Models
Paper
•
2311.08105
•
Published
•
16
NEVIS'22: A Stream of 100 Tasks Sampled from 30 Years of Computer Vision Research
Paper
•
2211.11747
•
Published
PODNet: Pooled Outputs Distillation for Small-Tasks Incremental Learning
Paper
•
2004.13513
•
Published
WARP: On the Benefits of Weight Averaged Rewarded Policies
Paper
•
2406.16768
•
Published
•
23
DiPaCo: Distributed Path Composition
Paper
•
2403.10616
•
Published
•
14
authored
a
paper
almost 2 years ago