Luce-Org/lucebox
View on GitHub ↗ ★ 2,981 C++
LLM speculative inference server for heterogeneous hardware & consumer GPUs
5 mentions on Hacker News by 3 people: 3 stories, 2 comments · first 2026-04-20 · latest 2026-06-16
20082026
-
▲ 3
256K context with 72MiB of KV cache on the GPUstory by GreenGames · 2026-06-16 · 0 comments
-
Something promising I found is "DFlash DDtree Qwen3.5 & Qwen3.6 27B GGUF on RTX 3090" - https://github.com/Luce-Org/lucebox-hub Didn't know 3.6 was available on Ollama outside of MacOS!
comment by gavmor · 2026-05-07 · in ZAYA1-8B matches DeepSeek-R1 on math with less than 1B active parameters -
▲ 3
PFlash: 10x prefill speedup over llama.cpp at 128K on a RTX 3090story by GreenGames · 2026-05-01 · 1 comment
-
…use it tactically in ways that are a net benefit to you. Not in ways that embarrass you or lie. * https://github.com/Luce-Org/lucebox-hub/commit/cfc38f67275ee... * * Here's Claude's version of this very post if you want to see an example…
comment by refulgentis · 2026-04-20 · in We got 207 tok/s with Qwen3.5-27B on an RTX 3090 -
▲ 165
We got 207 tok/s with Qwen3.5-27B on an RTX 3090story by GreenGames · 2026-04-20 · 52 comments