Hacker GitHub

← All repos

karpathy/llm.c

View on GitHub ↗ ★ 31,368 Cuda

LLM training in simple, raw C/CUDA

28 mentions on Hacker News by 23 people: 9 stories, 19 comments · first 2024-04-08 · latest 2026-08-15

Mentions per year 28 mentions · busiest year 2024 (22)
20082026
AllStoriesComments NewestTopOldest
  1. https://github.com/karpathy/llm.c However, I wonder about the use of this. Typically, you'd want to use a GPU for inference, and programming those in C is quite a challenge. The educational value of learning LLMs in C/C++ seems a lot lower…
    comment by smokel · 2026-08-15 · in AI by Hand
  2. https://github.com/karpathy/nanochat https://github.com/karpathy/llm.c The proof is in the pudding. Let's see your code
    comment by reducesuffering · 2026-01-27 · in A few random notes from Claude coding quite a bit last few weeks
  3. …to start from scratch and spend my time watching/reading? Is it along the same lines as https://github.com/karpathy/llm.c/discussions/677 ? He (karpathy) has a video series that also does something similar. I found it very informative and…
    comment by lacoolj · 2025-12-09 · in LLM from scratch, part 28 – training a base model from scratch on an RTX 3090
  4. …[0] I mean the first Mamba paper, not the first SSM paper btw: https://arxiv.org/abs/2312.00752 [1] https://github.com/karpathy/llm.c/discussions/677 [2] https://arxiv.org/abs/2106.12423 [3] https://www.ft.com/content/d5f91c27-3be8-454a-bea…
    comment by godelski · 2025-08-08 · in GPT-5: Key characteristics, pricing and system card
  5. Hello, I've worked on re-writing Karpathy's amazing work, llm.c ( https://github.com/karpathy/llm.c ), in Rust ( https://github.com/Steboss/llm.rust ). The overall initial performance, on a small dataset (the tiny Shakespeare) is very…
    comment by steboss · 2025-01-22 · in LLM training written in Rust, 30 times better than C implementation
  6. Using web wrappers around AI is being an AI end user If you want to be an AI engineer study this: https://github.com/karpathy/llm.c And build around llama.cpp Ollama is like cpanel for models. It’s not going to familiarize you with lower…
    comment by ohgrrr · 2024-11-27 · in Ask HN: Recommendation for a SWE looking to get up to speed with latest on AI
  7. …You can podcastify anything . I give it train_gpt2.c (C code that trains GPT-2): https://github.com/karpathy/llm.c/blob/master/train_gpt2.c and made a podcast about that: https://notebooklm.google.com/notebook/2585c187-b059-475a-b4... I do
    comment by lupire · 2024-09-29 · in NotebookLM is quite powerful and worth playing with
  8. …to create a minimal example of quadrotor control and planning in Rust. I was inspired by [llm.c]( https://github.com/karpathy/llm.c ), which teaches complex topics with minimal code. My goal was to provide a similar experience, introducing…
    comment by yangrobotics · 2024-08-26 · in Peng – A minimal Rust-based quadrotor simulation pipeline
  9. …this seems to be about the same order of magnitude compute as Karpathy's recent GPT-2 work: https://github.com/karpathy/llm.c/discussions/677 You could take the final checkpoint from that page and run it for some additional steps and see…
    comment by turtles3 · 2024-07-13 · in So you want to rent an NVIDIA H100 cluster? 2024 Consumer Guide
  10. See also (from the same author) https://github.com/karpathy/llm.c — "LLMs in simple, pure C/CUDA with no need for 245MB of PyTorch or 107MB of cPython."
    comment by imurray · 2024-06-11 · in NanoGPT: The simplest, fastest repository for training medium-sized GPTs
  11. …quite compute constrained. Training a GPT-2 sized model costs ~$20 nowadays in respect to compute: https://github.com/karpathy/llm.c/discussions/481
    comment by krasin · 2024-06-03 · in Grokfast: Accelerated Grokking by Amplifying Slow Gradients
  12. …is a magician! But being the coolest kid on the block with pure C/CUDA implementation is not enough https://github.com/karpathy/llm.c Studying a baby Llama 2 model source code in pure Mojo is the next level…
    comment by adeptima · 2024-05-29 · in Reproducing GPT-2 in llm.c
  13. Well here is a comment on 4090 https://github.com/karpathy/llm.c/discussions/481#discussion...
    comment by sabareesh · 2024-05-28 · in Reproducing GPT-2 in llm.c
  14. …that can host the 135GB of tokens for FineWeb100B. Will probably use S3 or something. Related see: https://github.com/karpathy/llm.c/issues/482
    comment by karpathy · 2024-05-28 · in Reproducing GPT-2 in llm.c
  15. Hi HN the main (more detailed) article is here https://github.com/karpathy/llm.c/discussions/481 Happy to answer questions!
    comment by karpathy · 2024-05-28 · in Reproducing GPT-2 in llm.c
  16. Have him check out: LLM training in simple, raw C/CUDA ---------------------------------- https://github.com/karpathy/llm.c It is only 1,000 lines of easy to read C code. There is also Python reference code.
    comment by icsa · 2024-04-25 · in Ask HN: 16 yo Nephew, in E. Africa, wants to train an LLM with on disk Wikipedia
  17. I'm not grasping the significance of this. I see a repo called llm.c, https://github.com/karpathy/llm.c , is the main selling point that PyTorch is a large download and this isn't... or are there other problems with PyTorch? I thought the…
    comment by politelemon · 2024-04-14 · in llm.c is now down to 26.2ms/iteration, matching PyTorch
  18. …- Highly recommend this one https://www.youtube.com/watch?v=PaCmpygFfXo - https://github.com/karpathy/llm.c - https://ig.ft.com/generative-ai 1: ex: https://news.ycombinator.com/reply?id=40007096&goto=item%3Fi...
    comment by georgehill · 2024-04-12 · in Show HN: Next-token prediction in JavaScript
  19. I wrote this, which might be a bit helpful: https://github.com/karpathy/llm.c/blob/master/doc/layernorm/... But if you don't have the background, I'd recommend my YouTube videos, see the Zero To Hero playlist:…
    comment by karpathy · 2024-04-08 · in Llm.c – LLM training in simple, pure C/CUDA