NousResearch/DisTrO
View on GitHub ↗ ★ 1,065
Distributed Training Over-The-Internet
8 mentions on Hacker News by 8 people: 4 stories, 4 comments · first 2024-08-26 · latest 2026-06-13
20082026
-
…compressed 10000x with the right tricks, I think it is achievable. Nous claims they did it already: https://github.com/NousResearch/DisTrO There are other gradient compression papers from the past reporting large compression rates
comment by WithinReason · 2026-06-13 · in Open source AI must win -
…Arthur seems to think test-time training will make his approach become the norm. sources: DisTrO: https://github.com/NousResearch/DisTrO DeMo: https://arxiv.org/pdf/2411.19870 Exo: https://github.com/exo-explore/exo
comment by aimanbenbaha · 2025-01-14 · in Training AI models might not need enormous data centres -
…way to improve on that using a method they call DisTro [2]. 1. https://arxiv.org/abs/2311.08105 2. https://github.com/NousResearch/DisTrO?tab=readme-ov-file
comment by bloatedGoat · 2024-09-18 · in Meta AI: "The Future of AI Is Open Source and Decentralized" -
…benefit from this about to be released optimizer that reduces intra-GPU communication [0]. [0] https://github.com/NousResearch/DisTrO/blob/main/A_Prelimina...
comment by jeroenvlek · 2024-08-28 · in Tesla’s TTPoE at Hot Chips 2024: Replacing TCP for Low Latency Applications