TheTom/llama-cpp-turboquant
View on GitHub ↗ ★ 2,445 C++
LLM inference in C/C++
5 mentions on Hacker News by 5 people: 0 stories, 5 comments · first 2026-03-28 · latest 2026-10-17
20082026
-
…results with that setup. Not sure what llamacpp build you are using but on mine using this fork: https://github.com/TheTom/llama-cpp-turboquant on commit 407f3237bfb3eeaff61546797de3d8c1a96be748 compiled for Pascal, I am able to push my…
comment by lopotubez · 2026-10-17 · in Show HN: Llmspeed – feel how fast a local LLM runs before you buy the hardware -
…to get a context of 195k for Qwen3.5-9b Q4_K_M using a llama.cpp fork that supports TurboQuant: https://github.com/TheTom/llama-cpp-turboquant I think you could replicate this with a larger model on your device. Overall with the right…
comment by Tade0 · 2026-10-04 · in The work by Valve's Timur Kristóf on improving old AMD GPUs on Linux -
…--cache-type-v turbo3 --no-mmap --mlock --host 0.0.0.0 -t 8 -tb 8 -np 1 Using this llama.cpp fork https://github.com/TheTom/llama-cpp-turboquant and mostly copying from this video https://www.youtube.com/watch?v=8F_5pdcD3HY Haven't had…
comment by unleaded · 2026-07-02 · in Kimi K2.7 Code is generally available in GitHub Copilot -
I built TurboQuant+ ( https://github.com/TheTom/llama-cpp-turboquant ), the llama.cpp implementation of this paper with extensions: asymmetric K/V compression, boundary layer protection, sparse V dequant, and this week weight compression…
comment by pidtom · 2026-04-03 · in Show HN: TurboQuant for vector search – 2-4 bit compression -
…Qwen3-8B-Q5_K_M-Instruct.gguf --local-dir ~/.models/` TurboQuant: cd ~ git clone https://github.com/TheTom/llama-cpp-turboquant.git cd llama-cpp-turboquant git checkout feature/turboquant-kv-cache Build for M4: cmake -B build \ -DGGML_META
comment by pow-tac · 2026-03-28 · in Qwen3 512k context via TurboQuant on Mac mini