Hacker GitHub

← All repos

google/gemma.cpp

View on GitHub ↗ ★ 7,075 C++

lightweight, standalone C++ inference engine for Google's Gemma models.

20 mentions on Hacker News by 10 people: 3 stories, 17 comments · first 2024-02-21 · latest 2026-08-24

Mentions per year 20 mentions · busiest year 2024 (17)
20082026
AllStoriesComments NewestTopOldest
  1. …help: https://gcc.godbolt.org/z/KM3ben7ET Maybe interesting to also refer to Gemma.cpp's version: https://github.com/google/gemma.cpp/blob/dev/ops/ops-inl.h#L... We recently added a FastExpMinusOrZero, only valid for inputs <= 0, which is…
    comment by janwas · 2026-08-24 · in SIMD Softmax, Neon to WASM, with Google Highway
  2. …LLM pair programmer and ask it to help you study the kernels in the ops/ library of gemma.cpp: https://github.com/google/gemma.cpp/tree/main/ops
    comment by austinvhuang · 2026-01-28 · in I have written gemma3 inference in pure C
  3. …to put the SIMD code, as well as the self-#include of itself, in a load.cxx TU. Here is an example: https://github.com/google/gemma.cpp/blob/9dfe2a76be63bcfe679...
    comment by janwas · 2025-01-09 · in Expressive Vector Engine – SIMD in C++
  4. This is in the works in the dev branch (thanks pchx :) https://github.com/google/gemma.cpp/pull/274
    comment by austinvhuang · 2024-06-27 · in Gemma 2: Improving Open Language Models at a Practical Size [pdf]
  5. Will gemma2 be available through gemma.cpp? https://github.com/google/gemma.cpp
    comment by jpcapdevila · 2024-06-27 · in Gemma 2: Improving Open Language Models at a Practical Size [pdf]
  6. Anyone interested in getting such numbers could run github.com/google/gemma.cpp on Arm hardware with hwy::DisableTargets(HWY_ALL_NEON) or HWY_ALL_SVE to compare the two :) I'd be curious to see the result. Calling hwy::DispatchedTarget…
    comment by janwas · 2024-05-25 · in Apple's M4 has reportedly adopted the ARMv9 architecture
  7. For anyone interested in a C++ implementation, our github.com/google/gemma.cpp now supports this model.
    comment by janwas · 2024-04-11 · in Implementation of Google's Griffin Architecture – RNN LLM
  8. …3D video in better than real time it's likely memory requirements will increase , not decrease. [1] https://github.com/google/gemma.cpp/issues/23
    comment by nl · 2024-04-03 · in PyTorch Library for Running LLM on Intel CPU and GPU
  9. For C++, also check out our https://github.com/google/gemma.cpp/blob/main/gemma.cc , which has direct calls to MatVec.
    comment by janwas · 2024-04-02 · in LLaMA now goes faster on CPUs
  10. Gemma.cpp was also affected, and now fixed ( https://github.com/google/gemma.cpp/pull/93 ). Thanks for the heads-up!
    comment by janwas · 2024-03-13 · in Fixing Gemma Bugs
  11. …E5M2 and E4M3 decodes to bf16 in ~14 instructions, so we can do that on the fly during dot products. [1]: github.com/google/gemma.cpp
    comment by janwas · 2024-03-13 · in Building Meta's GenAI infrastructure
  12. Looks like they're working on it: https://github.com/google/gemma.cpp/issues/16
    comment by jvolkman · 2024-02-24 · in Gemma.cpp: lightweight, standalone C++ inference engine for Gemma models
  13. …inspired by vertically-integrated model implementations such as ggml, llama.c, and llama.rs. from https://github.com/google/gemma.cpp
    comment by htrp · 2024-02-24 · in Understanding, using, and finetuning Gemma
  14. …for pointing that out. The README is being updated, you can see an updated WIP in the dev branch: https://github.com/google/gemma.cpp/tree/dev?tab=readme-ov-f... and improving error messages is a high priority. The weights should be the…
    comment by austinvhuang · 2024-02-23 · in Gemma.cpp: lightweight, standalone C++ inference engine for Gemma models
  15. ▲ 422
    Gemma.cpp: lightweight, standalone C++ inference engine for Gemma models
    story by mfiguiere · 2024-02-23 · 130 comments
  16. ▲ 3
    Gemma.cpp: lightweight, standalone C++ inference engine for Gemma models
    story by ot · 2024-02-21 · 0 comments
  17. …we also implemented a standalone C++ implementation that runs locally with just CPU simd https://github.com/google/gemma.cpp
    comment by austinvhuang · 2024-02-21 · in Gemma: New Open Models
  18. …ggml as options, we also implemented a standalone C++ implementation that you can run locally at https://github.com/google/gemma.cpp
    comment by austinvhuang · 2024-02-21 · in Gemma: New Open Models
  19. ▲ 4
    Source code of Google Gemma model in C++
    story by yu3zhou4 · 2024-02-21 · 0 comments
  20. They have implemented the model also on their own C++ inference engine: https://github.com/google/gemma.cpp
    comment by GaggiX · 2024-02-21 · in Gemma: New Open Models