Hacker GitHub

← All repos

ggml-org/llama.cpp

View on GitHub ↗ ★ 133,134 C++

LLM inference in C/C++

738 mentions on Hacker News by 471 people: 96 stories, 642 comments · first 2023-03-10 · latest 2026-10-28

Mentions per year 738 mentions · busiest year 2023 (316)
20082026
AllStoriesComments NewestTopOldest
  1. ▲ 100
    Performance of llama.cpp on Apple Silicon A-series
    story by mobilio · 2023-12-19 · 41 comments
  2. ▲ 3
    Llama.cpp: Support for Phi-2
    story by tosh · 2023-12-19 · 0 comments
  3. ▲ 3
    QMoE Support for Mixtral
    story by tosh · 2023-12-14 · 0 comments
  4. ▲ 2
    Llama: Add Mixtral Support
    story by tosh · 2023-12-11 · 0 comments
  5. ▲ 2
    Performance of Llama.cpp on Apple Silicon
    story by tosh · 2023-11-29 · 0 comments
  6. ▲ 1
    Adjust VRAM/RAM Split on Apple Silicon
    story by tosh · 2023-11-29 · 1 comment
  7. ▲ 96
    Running Llama.cpp on AWS Instances
    story by schappim · 2023-11-27 · 10 comments
  8. ▲ 2
    (2) Apple Silicon Performance · ggerganov/llama.cpp · Discussion #4167
    story by gavi · 2023-11-26 · 0 comments
  9. ▲ 1
    Llama on Mac M2 Ultra (Literally)
    story by behnamoh · 2023-11-10 · 0 comments
  10. ▲ 4
    Show HN: I built BakLLaVA and llama.cpp demo and it went viral on X
    story by Obertr · 2023-11-05 · 1 comment
  11. ▲ 2
    LLM quantization severely damages model quality and perplexity
    story by behnamoh · 2023-10-20 · 3 comments
  12. ▲ 2
    Apple predicted the rise of local LLMs, hence the M2 Ultra
    story by behnamoh · 2023-10-13 · 3 comments
  13. ▲ 8
    gg: "M2 Ultra is the absolute best personal LLM inference node you can buy."
    story by behnamoh · 2023-10-12 · 0 comments
  14. ▲ 268
    M2 Ultra can run 128 streams of Llama 2 7B in parallel
    story by behnamoh · 2023-10-11 · 173 comments
  15. ▲ 2
    Llama.cpp Was Hacked in an Evening
    story by behnamoh · 2023-10-11 · 0 comments
  16. ▲ 1
    I got llama.cpp and StarCoder – 1B to run on my P4 Retro PC
    story by vkaku · 2023-10-01 · 1 comment
  17. ▲ 6
    llama.cpp now supports StarCoder model series
    story by wsxiaoys · 2023-09-18 · 1 comment
  18. ▲ 4
    Llama.cpp speculative sampling: 2x faster inference for large models
    story by bobivl · 2023-09-05 · 1 comment
  19. ▲ 16
    Speculative: PoC for speeding-up inference via speculative sampling by ggerganov
    story by kristianp · 2023-09-02 · 1 comment
  20. ▲ 2
    Llama.cpp Supports Falcon Now
    story by gslin · 2023-08-25 · 0 comments
  21. ▲ 4
    AMD ROCm Support Added to Llama.cpp
    story by irusensei · 2023-08-25 · 0 comments
  22. ▲ 2
    New llama.cpp format GGUF now merged
    story by mchiang · 2023-08-21 · 0 comments
  23. ▲ 2
    GPU Support to Ggml
    story by melenaboija · 2023-08-19 · 0 comments
  24. ▲ 15
    Ask HN: Are you training and running custom LLMs and how are you doing it?
    story by kordlessagain · 2023-08-14 · 2 comments
  25. ▲ 19
    Show HN: Grammar Generator App for Llama.cpp
    story by aduffy · 2023-08-02 · 6 comments
  26. ▲ 417
    Llama: Add grammar-based sampling
    story by davepeck · 2023-07-21 · 105 comments
  27. ▲ 3
    Llama 2: poc for running 70B on CPU
    story by tosh · 2023-07-19 · 0 comments
  28. ▲ 328
    LLama.cpp now has a web interface
    story by xal · 2023-07-05 · 49 comments
  29. ▲ 1
    Llama.cpp now has a simple web UI for chat
    story by wsgeorge · 2023-07-04 · 0 comments
  30. ▲ 3
    Inference at the edge: Efficient transformer model inference on-device
    story by lioeters · 2023-07-02 · 0 comments
  31. ▲ 121
    Show HN: Project S.A.T.U.R.D.A.Y. – open-source, self hosted, J.A.R.V.I.S.
    story by GRVYDEV · 2023-07-02 · 30 comments
  32. ▲ 24
    WIP Llama.cpp Vulkan Implementations
    story by brucethemoose2 · 2023-07-01 · 1 comment
  33. ▲ 728
    Llama.cpp: Full CUDA GPU Acceleration
    story by gzer0 · 2023-06-13 · 310 comments
  34. ▲ 3
    K-Quants
    story by tosh · 2023-06-10 · 0 comments
  35. ▲ 1
    Llama.cpp: Roadmap June 2023
    story by tosh · 2023-06-07 · 0 comments
  36. ▲ 4
    Full GPU Inference of LLaMA on Apple Silicon Using Metal
    story by behnamoh · 2023-06-05 · 0 comments
  37. ▲ 97
    llama.cpp: Roadmap May 2023
    story by tosh · 2023-04-30 · 6 comments
  38. ▲ 3
    Llama.cpp: Add GPU support to ggml
    story by mromanuk · 2023-04-12 · 0 comments
  39. ▲ 37
    Show HN: TurboPilot: Copilot clone runs code completion LLM on your CPU
    story by DrRavenstein · 2023-04-10 · 4 comments
  40. ▲ 86
    Revert for jart’s llama.cpp MMAP miracles
    story by mmoustafa · 2023-04-02 · 86 comments
  41. ▲ 31
    30B model now needs only 5.8GB of RAM? How?
    story by olalonde · 2023-04-01 · 11 comments
  42. ▲ 1,311
    Llama.cpp 30B runs with only 6GB of RAM now
    story by msoad · 2023-03-31 · 414 comments
  43. ▲ 2
    Inference at the Edge
    story by georgehill · 2023-03-17 · 0 comments
  44. ▲ 4
    Inference at the Edge
    story by Mizza · 2023-03-16 · 0 comments
  45. ▲ 6
    LLaMA 7B model running on 4GB RAM Raspberry Pi 4
    story by amrrs · 2023-03-12 · 0 comments
  46. ▲ 989
    Llama.cpp: Port of Facebook's LLaMA model in C/C++, with Apple Silicon support
    story by mrtksn · 2023-03-10 · 284 comments