b4rtaz/distributed-llama
View on GitHub ↗ ★ 3,125 C++
Distributed LLM inference. Connect home devices into a powerful cluster to accelerate LLM inference. More devices means faster inference.
16 mentions on Hacker News by 10 people: 11 stories, 5 comments · first 2024-01-20 · latest 2026-02-12
20082026
-
There's a couple of alternatives to exo it seems https://github.com/b4rtaz/distributed-llama and https://github.com/ray-project/ray
comment by fouc · 2025-08-09 · in I want everything local – Building my offline AI workspace -
https://github.com/b4rtaz/distributed-llama ?
comment by yjftsjthsd-h · 2025-08-07 · in Benchmark Framework Desktop Mainboard and 4-node cluster -
…https://github.com/mozilla-Ocho/llamafile https://justine.lol/matmul/ and parallelize them with https://github.com/b4rtaz/distributed-llama Obviously this needs some knowledge of the command line, so get a good terminal emulator like…
comment by LargoLasskhyfv · 2024-09-21 · in Ask HN: How can I experiment with LLMs with a old machine? -
…a bit of info on the project page. It's CPU only, which is why GPU and VRAM is never mentioned: https://github.com/b4rtaz/distributed-llama I'm using an old x99 board for my desktop currently. If I swap out the i7 with a Xeon it can take…
comment by duffyjp · 2024-07-29 · in How to Run Llama 3 405B on Home Devices? Build AI Cluster