← All repos

omgitsbase/llmash

View on GitHub ↗ ★ 37 C++

Ollama-compatible LLM server on a tuned llama.cpp fork. Beats vllm by 40% on single stream decode, and by 20% on multi stream decode, and beats ollama by much 2-4x. Made possible with custom CUDA kernels, MTP, speculation and its own 3-bit builds. Uncensors models in one command. Windows and Linux, NVIDIA. Optimized to not eat up ram.

4 mentions on Hacker News by 1 person: 4 stories, 0 comments · first 2026-09-07 · latest 2026-09-09