google/gemma.cpp
View on GitHub ↗ ★ 7,075 C++
lightweight, standalone C++ inference engine for Google's Gemma models.
20 mentions on Hacker News by 10 people: 3 stories, 17 comments · first 2024-02-21 · latest 2026-08-24
20082026
-
…help: https://gcc.godbolt.org/z/KM3ben7ET Maybe interesting to also refer to Gemma.cpp's version: https://github.com/google/gemma.cpp/blob/dev/ops/ops-inl.h#L... We recently added a FastExpMinusOrZero, only valid for inputs <= 0, which is…
comment by janwas · 2026-08-24 · in SIMD Softmax, Neon to WASM, with Google Highway -
…LLM pair programmer and ask it to help you study the kernels in the ops/ library of gemma.cpp: https://github.com/google/gemma.cpp/tree/main/ops
comment by austinvhuang · 2026-01-28 · in I have written gemma3 inference in pure C -
…to put the SIMD code, as well as the self-#include of itself, in a load.cxx TU. Here is an example: https://github.com/google/gemma.cpp/blob/9dfe2a76be63bcfe679...
comment by janwas · 2025-01-09 · in Expressive Vector Engine – SIMD in C++ -
This is in the works in the dev branch (thanks pchx :) https://github.com/google/gemma.cpp/pull/274
comment by austinvhuang · 2024-06-27 · in Gemma 2: Improving Open Language Models at a Practical Size [pdf] -
Will gemma2 be available through gemma.cpp? https://github.com/google/gemma.cpp
comment by jpcapdevila · 2024-06-27 · in Gemma 2: Improving Open Language Models at a Practical Size [pdf] -
Anyone interested in getting such numbers could run github.com/google/gemma.cpp on Arm hardware with hwy::DisableTargets(HWY_ALL_NEON) or HWY_ALL_SVE to compare the two :) I'd be curious to see the result. Calling hwy::DispatchedTarget…
comment by janwas · 2024-05-25 · in Apple's M4 has reportedly adopted the ARMv9 architecture -
For anyone interested in a C++ implementation, our github.com/google/gemma.cpp now supports this model.
comment by janwas · 2024-04-11 · in Implementation of Google's Griffin Architecture – RNN LLM -
…3D video in better than real time it's likely memory requirements will increase , not decrease. [1] https://github.com/google/gemma.cpp/issues/23
comment by nl · 2024-04-03 · in PyTorch Library for Running LLM on Intel CPU and GPU -
For C++, also check out our https://github.com/google/gemma.cpp/blob/main/gemma.cc , which has direct calls to MatVec.
comment by janwas · 2024-04-02 · in LLaMA now goes faster on CPUs -
Gemma.cpp was also affected, and now fixed ( https://github.com/google/gemma.cpp/pull/93 ). Thanks for the heads-up!
comment by janwas · 2024-03-13 · in Fixing Gemma Bugs -
…E5M2 and E4M3 decodes to bf16 in ~14 instructions, so we can do that on the fly during dot products. [1]: github.com/google/gemma.cpp
comment by janwas · 2024-03-13 · in Building Meta's GenAI infrastructure -
Looks like they're working on it: https://github.com/google/gemma.cpp/issues/16
comment by jvolkman · 2024-02-24 · in Gemma.cpp: lightweight, standalone C++ inference engine for Gemma models -
…inspired by vertically-integrated model implementations such as ggml, llama.c, and llama.rs. from https://github.com/google/gemma.cpp
comment by htrp · 2024-02-24 · in Understanding, using, and finetuning Gemma -
…for pointing that out. The README is being updated, you can see an updated WIP in the dev branch: https://github.com/google/gemma.cpp/tree/dev?tab=readme-ov-f... and improving error messages is a high priority. The weights should be the…
comment by austinvhuang · 2024-02-23 · in Gemma.cpp: lightweight, standalone C++ inference engine for Gemma models -
…we also implemented a standalone C++ implementation that runs locally with just CPU simd https://github.com/google/gemma.cpp
comment by austinvhuang · 2024-02-21 · in Gemma: New Open Models -
…ggml as options, we also implemented a standalone C++ implementation that you can run locally at https://github.com/google/gemma.cpp
comment by austinvhuang · 2024-02-21 · in Gemma: New Open Models -
They have implemented the model also on their own C++ inference engine: https://github.com/google/gemma.cpp
comment by GaggiX · 2024-02-21 · in Gemma: New Open Models