Hacker GitHub

← All repos

PrismML-Eng/llama.cpp

View on GitHub ↗ ★ 1,009 C++

LLM inference in C/C++

6 mentions on Hacker News by 5 people: 0 stories, 6 comments · first 2026-04-01 · latest 2026-09-18

Mentions per year 6 mentions · busiest year 2026 (6)
20082026
AllStoriesComments NewestTopOldest
  1. …model card with minor modifications (modify the architecture 86 for your own hardware): git clone https://github.com/PrismML-Eng/llama.cpp && cd llama.cpp cmake -B build -DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=86 && cmake --build build…
    comment by wombat23 · 2026-09-18 · in Bonsai 2 27B: Near-Lossless Compression in a 9x Smaller Footprint
  2. …be aware that you need Prism's llama.cpp fork to get them to work, from https://github.com/PrismML-Eng/llama.cpp/releases/tag/prism-... This should work: cd /tmp # Get the Prism macOS runtime curl -fL https://github.com/PrismML-En
    comment by simonw · 2026-09-17 · in Bonsai 2 27B: Near-Lossless Compression in a 9x Smaller Footprint
  3. I got their previous model working in their custom fork of llama.cpp ( https://github.com/PrismML-Eng/llama.cpp ). I haven't tried this one yet, but will find some time to benchmark it sometime this week. Though this says mainline…
    comment by SwellJoe · 2026-07-14 · in Bonsai 27B: A 27B-Class model that runs on a phone
  4. …8 \ --cache-type-k q8_0 \ --cache-type-v q8_0 \ --log-colors on # llama.cpp is forked one: https://github.com/PrismML-Eng/llama.cpp.git # The server can serve 5 parallel request, with each request capped at around `13K` tokens... # A bit…
    comment by freakynit · 2026-04-21 · in Ternary Bonsai: Top Intelligence at 1.58 Bits
  5. prismML provides a llama.cpp fork which is compatible with the 1 bit models: https://github.com/PrismML-Eng/llama.cpp After fails with Ollama and main llama.cpp the fork worked on my M5 MBA. Edit: Typos
    comment by m0do1 · 2026-04-01 · in Show HN: 1-Bit Bonsai, the First Commercially Viable 1-Bit LLMs
  6. …of outputs is on par with top models of similar size. 1. Clone their forked repo: `git clone https://github.com/PrismML-Eng/llama.cpp.git ` 2. Then (assuming you already have xcode build tools installed): cd llama.cpp cmake -B build…
    comment by freakynit · 2026-04-01 · in Show HN: 1-Bit Bonsai, the First Commercially Viable 1-Bit LLMs