tensorninja/ninfer-4090
View on GitHub ↗ ★ 16 C++
From-scratch C++/CUDA engine for Qwen3.8-27B on one RTX 4090: 3,600 tok/s prefill, 149 tok/s MTP decode, full 262K context. Works with any coding agent over OpenAI and Anthropic APIs, best with Caudra. Per-request LoRA adapters, plus OpenAI Decisions and TypeSafe System One on the same weights.
4 mentions on Hacker News by 1 person: 0 stories, 4 comments · first 2026-08-26 · latest 2026-11-05
20082026
-
…building this into Caudra as well so it can be turned into more like a swarm of agents [3]. [0] https://github.com/tensorninja/ninfer-4090 [1] https://caudra.ai/features/decisions/ [2] https://github.com/caudra/caudra [3]…
comment by netsroht · 2026-11-05 · in Ask HN: What's your current AI agentic setup? -
…fork [0] if there's a benefit to it. But sometimes I think it's just more art that science :) [0] https://github.com/tensorninja/ninfer-4090
comment by netsroht · 2026-11-04 · in Qwen 3.8 27B on my laptop can interpret model weights, too -
…yourself. I should probably also add support for this new openai decisions API format as well. [1] https://github.com/tensorninja/ninfer-4090
comment by netsroht · 2026-10-07 · in Decisions API is in public beta -
…night when the sun doesn't shine (even with a battery) I'd like to limit this a little bit. [0] https://github.com/tensorninja/ninfer-4090
comment by netsroht · 2026-08-26 · in Apple introduces M6 and M5 Ultra