jmaczan/tiny-vllm
View on GitHub ↗ ★ 1,185 C++
Build your own high performance LLM inference engine in C++ and CUDA - a smaller version of vLLM
6 mentions on Hacker News by 1 person: 2 stories, 4 comments · first 2026-03-09 · latest 2026-08-07
20082026
-
…prefer C++ and CUDA, then there's also tiny-vllm of mine [0] - recently we broke 1k gh stars [0] https://github.com/jmaczan/tiny-vllm
comment by yu3zhou4 · 2026-08-07 · in Inside vLLM: Anatomy of a High-Throughput LLM Inference System (2025) -
An open course on building high performance LLM inference engine! Hope to finish by the end of April https://github.com/jmaczan/tiny-vllm
comment by yu3zhou4 · 2026-04-12 · in Ask HN: What Are You Working On? (April 2026) -
…much shorter and not that deep as this one, for anyone interested in topic you might like it too https://github.com/jmaczan/tiny-vllm?tab=readme-ov-file#how-...
comment by yu3zhou4 · 2026-04-07 · in Floating point from scratch: Hard Mode -
…a tiny version of vLLM in C++ and CUDA from scratch (high throughput LLM inference server) https://github.com/jmaczan/tiny-vllm
comment by yu3zhou4 · 2026-03-09 · in Ask HN: What Are You Working On? (March 2026)