SPICE: Sparse Parameter Gossip for LLM Pretrainingwith Pietro Cagnasso, Edouard Oyallon, Eugene Belilovsky1 versionW_hg6sdk7pv1 · current