Hacker GitHub

← All repos

alainnothere/llama.cpp

View on GitHub ↗ ★ 1 C++

LLM inference in C/C++

8 mentions on Hacker News by 1 person: 2 stories, 6 comments · first 2026-08-16 · latest 2026-11-08

Mentions per year 8 mentions · busiest year 2026 (8)
20082026
AllStoriesComments NewestTopOldest
  1. ▲ 2
    Qwen3.8-27B MTP Q4S head on Q6 quant is faster than q6, f16 slower = acceptance
    story by xlayn · 2026-11-08 · 1 comment
  2. …kv to disk so you can resume I keep this branch of llama.cpp that includes that same functionality https://github.com/alainnothere/llama.cpp/commits/disk-cache... And you know it's load bearing each of the load baerings parts that bear…
    comment by xlayn · 2026-10-03 · in From the creator of Redis; run LLM locally with ds4
  3. For the impatient, I merged llama.cpp tentative branches to get it running here https://github.com/alainnothere/llama.cpp/tree/disk-cache-ev... , thing runs at 23.54 token/sec and my setup runs at high 30 the 3.8 dense 27B. and this is the…
    comment by xlayn · 2026-08-26 · in Qwen3.8-Flash-Next
  4. …you use a gguf, you can extract the template from the Qwen3.827B model using the following script https://github.com/alainnothere/llama.cpp/blob/disk-cache-ev... , use it like ggufGetTemplate.py MODEL.gguf [OUT.jinja] modify the template…
    comment by xlayn · 2026-08-19 · in Qwen3.8-27B make medium the default effort level instead of xhigh
  5. ▲ 16
    Qwen3.8-27B make medium the default effort level instead of xhigh
    story by xlayn · 2026-08-19 · 4 comments
  6. …be something private that you can use locally, I've been also working on this branch of llama.cpp https://github.com/alainnothere/llama.cpp/tree/disk-cache-ev... that I use along with privibe that stores conversations to disk, so if you…
    comment by xlayn · 2026-08-18 · in Privibe – LLM CLI Local first+privacy focus+llama.cpp cache branch and Qwen3.x
  7. …to disk so you can resume quickly days after) also accept the reasoning effort flag here https://github.com/alainnothere/llama.cpp/tree/disk-cache-ev... I did testing and the reasoning effort can be set per message, I was not aware of the…
    comment by xlayn · 2026-08-17 · in Qwen 3.8 27B is excellent, but it defaults to overthinking things
  8. …be something private that you can use locally, I've been also working on this branch of llama.cpp https://github.com/alainnothere/llama.cpp/tree/disk-cache-ev... that I use along with privibe that stores conversations to disk, so if you…
    comment by xlayn · 2026-08-16 · in Privibe - LLM Cli Local first+privacy focus+llama.cpp cache branch + Qwen3.x