Hacker GitHub

← All repos

b4rtaz/distributed-llama

View on GitHub ↗ ★ 3,125 C++

Distributed LLM inference. Connect home devices into a powerful cluster to accelerate LLM inference. More devices means faster inference.

16 mentions on Hacker News by 10 people: 11 stories, 5 comments · first 2024-01-20 · latest 2026-02-12

Mentions per year 16 mentions · busiest year 2024 (9)
20082026
AllStoriesComments NewestTopOldest
  1. ▲ 3
    Distributed Llama
    story by oldfuture · 2026-02-12 · 0 comments
  2. ▲ 347
    Qwen3 30B A3B Hits 13 token/s on 4xRaspberry Pi 5
    story by b4rtazz · 2025-09-06 · 161 comments
  3. There's a couple of alternatives to exo it seems https://github.com/b4rtaz/distributed-llama and https://github.com/ray-project/ray
    comment by fouc · 2025-08-09 · in I want everything local – Building my offline AI workspace
  4. https://github.com/b4rtaz/distributed-llama ?
    comment by yjftsjthsd-h · 2025-08-07 · in Benchmark Framework Desktop Mainboard and 4-node cluster
  5. ▲ 6
    Run Llama 3.3 70B Q40 on $1516 GPU 3.3 tok/s
    story by b4rtazz · 2025-04-26 · 0 comments
  6. ▲ 4
    Distributed-Llama: Connect home devices into a cluster for LLM inference
    story by tosh · 2025-02-15 · 1 comment
  7. ▲ 306
    Deepseek R1 Distill 8B Q40 on 4 x Raspberry Pi 5
    story by b4rtazz · 2025-02-15 · 156 comments
  8. …https://github.com/mozilla-Ocho/llamafile https://justine.lol/matmul/ and parallelize them with https://github.com/b4rtaz/distributed-llama Obviously this needs some knowledge of the command line, so get a good terminal emulator like…
    comment by LargoLasskhyfv · 2024-09-21 · in Ask HN: How can I experiment with LLMs with a old machine?
  9. …a bit of info on the project page. It's CPU only, which is why GPU and VRAM is never mentioned: https://github.com/b4rtaz/distributed-llama I'm using an old x99 board for my desktop currently. If I swap out the i7 with a Xeon it can take…
    comment by duffyjp · 2024-07-29 · in How to Run Llama 3 405B on Home Devices? Build AI Cluster
  10. Or maybe using Distributed Llama? https://github.com/b4rtaz/distributed-llama
    comment by niutech · 2024-07-26 · in Llama 3.1
  11. ▲ 4
    Llama 3.1 is available for home AI clusters! Run 8B model with the full context
    story by b4rtazz · 2024-07-25 · 0 comments
  12. ▲ 2
    Run Llama 3 405B on home cluster (Distributed Llama)
    story by b4rtazz · 2024-07-23 · 1 comment
  13. ▲ 1
    Distributed Grok-1 (314B)
    story by b4rtazz · 2024-04-15 · 0 comments
  14. ▲ 12
    Show HN: Distributed Llama – Run LLMs on multiple devices in parallel
    story by b4rtazz · 2024-01-25 · 0 comments
  15. ▲ 4
    Distributed Llama
    story by lagniappe · 2024-01-21 · 0 comments
  16. ▲ 4
    Distributed Llama on Raspberry Pis
    story by politelemon · 2024-01-20 · 2 comments