Helldez/BigMoeOnEdge
View on GitHub ↗ ★ 740 C++
Run MoE models bigger than your RAM. Frontier-size MoE on a phone, laptop, edge. Lossless, on stock llama.cpp. An adaptive streaming optimizer. Any hardware, any OS, any model, any quantization, behind a local OpenAI-compatible server.
4 mentions on Hacker News by 4 people: 2 stories, 2 comments · first 2026-07-19 · latest 2026-10-20
20082026
-
▲ 1
Show HN: A layer on top llama.cpp enabling big MoE streaming on phone and laptopstory by HelldezNoBan · 2026-10-20 · 0 comments
-
One with 12GB of RAM apparently: https://github.com/Helldez/BigMoeOnEdge
comment by latentsea · 2026-09-16 · in AI 'kill switch' may need to be mandatory, Anthropic co-founder tells BBC -
See also: https://github.com/Helldez/BigMoeOnEdge
comment by lern_too_spel · 2026-08-04 · in Show HN: Run an 80B Qwen in 4.3 GB of RAM on a Mac, and a 35B on an iPhone -
▲ 1
Show HN: Run a 120B-parameter MoE on Android mid-range phone CPU-only llama.cppstory by Helldez · 2026-07-19 · 0 comments