Neroued/ninfer
View on GitHub ↗ ★ 3,233 C++
High-performance single-GPU inference for selected model checkpoints and GPUs.
12 mentions on Hacker News by 10 people: 3 stories, 9 comments · first 2026-07-23 · latest 2026-10-04
20082026
-
▲ 1
High-performance single-GPU inference for selected model checkpoints and GPUsstory by davedx · 2026-08-15 · 0 comments
-
▲ 3
Ninfer: High-performance single-GPU inferencestory by handfuloflight · 2026-07-28 · 0 comments
-
▲ 2
NInfer: 271 tok/s (543 with speculative) for Qwen3.6-35B-A3B on one RTX 5090story by opwizardx · 2026-07-23 · 0 comments