ggml-org/ggml on Hacker News

…of llama.cpp and it's almost ready, but not quite. More info about the versioning process is here: https://github.com/ggml-org/ggml/discussions/1579

comment by ggerganov · 2026-08-17

…really easy, educational and fun to use. It is written in C and the backbone for llama.cpp. [ggml] https://github.com/ggml-org/ggml

comment by bssrdf · 2026-07-14 · in Tensor Is the Might

It’s a different repo. https://github.com/ggml-org/ggml The models are implemented by Ollama https://github.com/ollama/ollama/tree/main/model/models I can say as a fact, for the gpt-oss model, we also implemented our own MXFP4 kernel.…

comment by mchiang · 2025-08-06 · in Ollama Turbo

…“Some of the development is currently happening in the llama.cpp and whisper.cpp repos” -- https://github.com/ggml-org/ggml

comment by cwillu · 2025-05-16 · in Ollama's new engine for multimodal models

…is contained in the model file, and no additional information needs to be provided by the user. https://github.com/ggml-org/ggml/blob/master/docs/gguf.md We literally just got rid of that multi file chaos only for ollama to add it back :/

comment by Havoc · 2025-04-30 · in Xiaomi MiMo Reasoning Model

Summary at https://github.com/ggml-org/ggml/pull/1103

comment by krackers · 2025-03-26 · in Heap-overflowing Llama.cpp to RCE

…by llama.cpp and ggml, name inspired by the name of the author ( https://github.com/ggerganov ): https://github.com/ggerganov/ggml/blob/master/docs/gguf.md He prefixes everything with “gg” (his initials). EDIT: Confirmed:…

comment by w4 · 2024-11-10 · in Everything I've learned so far about running local LLMs

Cool. I was just learning about GGUF by creating my own parser for it based on the spec https://github.com/ggerganov/ggml/blob/master/docs/gguf.md (for educational purposes)

comment by null_point · 2024-03-01 · in GGUF, the Long Way Around

…you look up model quantization and some libraries like ggml[1] that actually do that for you. [1] https://github.com/ggerganov/ggml

comment by manca · 2024-02-27 · in Ask HN: People who switched from GPT to their own models. How was it?

…things like cuBLAS and I'd assume MKL too, which at best, can only operate on f32 and f16. https://github.com/ggerganov/ggml/issues/293 My concern with MKL is also that, judging by the llama.cpp README's brief mention of using it, adding…

comment by jart · 2024-01-01 · in Emacs-copilot: Large language model code completion for Emacs

Apparently this guy like a bunch of others like https://github.com/ggerganov/ggml are implementing transformers from papers for people that want them. Pretty cool.

comment by itissid · 2023-11-30 · in Q-Transformer

ggml.cpp with blast backend could be one example of it See for instance: https://github.com/ggerganov/ggml/blob/57c468b8655f3630d1749... which are the parts not available in blast

comment by Kubuxu · 2023-11-23 · in Exponentially faster language modelling

https://github.com/ggerganov/ggml TinyGrad is also targeting CPU inference, and IIRC it works ok in Apache TVM. One note is that prompt ingestion is extremely slow on CPU compared to GPU. So short prompts are fine (and tokens can be…

comment by brucethemoose2 · 2023-09-06 · in Falcon 180B

…After that, both ggml_vec_dot_f16 and ggml_vec_dot_f32 took first place in profiler. Source: https://github.com/ggerganov/ggml/blob/master/src/ggml.c#L10... But I agree, that in theory, and only with AVX512 BF16 (not exactly FP16, but…

comment by Lockal · 2023-08-19 · in Stable Diffusion in C/C++

…For anyone on HN, they should probably be looking at https://github.com/ggerganov/llama.cpp and https://github.com/ggerganov/ggml directly. If you have a high-end Nvidia consumer card (3090/4090) I'd highly recommend looking into…

comment by lhl · 2023-07-19 · in Accessing Llama 2 from the command-line with the LLM-replicate plugin

…>ggml-tiny.en.bin, ggml-base.en.bin, ggml-small.en.bin This uses Gerganov's Whisper models ( https://github.com/ggerganov/ggml )

comment by bertman · 2023-07-02 · in Show HN: bigwav.app – Transcription and Annotation using just a web browser

Mosaic's MPT models are already supported in GGML: https://github.com/ggerganov/ggml Here's MPT-30B running in 4-bit precision on CPU :) https://twitter.com/abacaj/status/1673133443339763712?s=20

comment by ml_hardware · 2023-06-26 · in Databricks Strikes $1.3B Deal for Generative AI Startup MosaicML

1 to 2 tokens per second on CPU, depending on the CPU/RAM, thanks to GGML. See discussions: https://github.com/ggerganov/ggml/pull/231 And the code: https://github.com/jploski/ggml/tree/falcon40b

comment by jxy · 2023-06-16 · in Falcon LLM: The Most Powerful Open Source LLM Released to Date

https://github.com/ggerganov/ggml/blob/master/src/ggml-openc...

comment by boringuser2 · 2023-06-07 · in GGML – AI at the Edge

It would be cool too see a port of llama.cpp and ggml to Zig https://github.com/ggerganov/llama.cpp https://github.com/ggerganov/ggml

comment by remorses · 2023-04-25 · in Deep Neural Networks from Scratch in Zig

it's probably simple for Dolly v1 (?) since it was a fine-tuned version of GPT-J https://github.com/ggerganov/ggml/tree/master/examples/gpt-j AFAIK there is no .cpp version of Pythia-12B yet

comment by anentropic · 2023-04-12 · in Databricks Releases 15K Record Training Corpus for Instruction Tuning LLMs

…written by Georgi Gerganov. It prioritizes inference on Apple hardware and low resource machines. https://github.com/ggerganov/ggml llama.cpp and alpaca.cpp (and other derivatives) all require model weights to be converted to the ggml…

comment by wsgeorge · 2023-03-29 · in Gpt4all: A chatbot trained on ~800k GPT-3.5-Turbo Generations based on LLaMa

Georgi rewrote the code on top of his own tensor library (ggml[0]). [0] https://github.com/ggerganov/ggml

comment by 19h · 2023-03-16 · in Show HN: Alpaca.cpp – Run an Instruction-Tuned Chat-Style LLM on a MacBook

ggml ( https://github.com/ggerganov/ggml ) has a GPT-J example, the 6B parameter model runs happily on the CPU 16gb of ram and 8 cores at a couple of words per second, no GPUs necessary. gptj_model_load: ggml ctx size = 13334.86 MB…

comment by quesomaster9000 · 2023-03-11 · in ChatGPT-J: The Privacy-First, Self-Hosted Chatbot Built on GPT-J's Powerful AI

…on a random HN comment earlier in a different topic where someone showed a GPT-J model on CPU only: https://github.com/ggerganov/ggml I tested it on my Linux and Macbook M1 Air and it generates tokens at a reasonable speed using CPU only.…

comment by adeon · 2023-02-20 · in Running large language models like ChatGPT on a single GPU

There's a float16 version of GPT-J rewritten in C++ here that apparently fits into 16GB... https://github.com/ggerganov/ggml/tree/master/examples/gpt-j it's CPU-only and runs inference at "about ~6 words per second" on an M1 MBP

comment by anentropic · 2023-02-14 · in ChatGPT Hardware: A look at 8x NVIDIA A100 powering the tool

…and GPT-J / GPT-JT (6B params) at 125ms/token. Here are a couple of generated examples using GPT-J: https://github.com/ggerganov/ggml/tree/master/examples/gpt-j These are examples using zero-shot prompt where the model auto-completes a…

comment by ggerganov · 2023-01-20 · in ChatRWKV, like ChatGPT but powered by the RWKV (RNN-based, open) language model

I'm curious how they chose between: A) ggml https://github.com/ggerganov/ggml/tree/master/examples/gpt-2 B) Fabrice Bellard's GPT2C https://bellard.org/libnc/gpt2tc.html

comment by rahimnathwani · 2022-12-07 · in Talk = GPT-2 and Whisper and WASM