Hacker GitHub

← All repos

karpathy/minGPT

View on GitHub ↗ ★ 24,994 Python

A minimal PyTorch re-implementation of the OpenAI GPT (Generative Pretrained Transformer) training

21 mentions on Hacker News by 20 people: 3 stories, 18 comments · first 2020-08-17 · latest 2026-04-06

Mentions per year 21 mentions · busiest year 2023 (8)
20082026
AllStoriesComments NewestTopOldest
  1. …to Andrej Karpathy's microgpt ( https://karpathy.github.io/2026/02/12/microgpt/ ) or minGPT ( https://github.com/karpathy/minGPT )?
    comment by fg137 · 2026-04-06 · in Show HN: I built a tiny LLM to demystify how language models work
  2. …from weight decay as standard practice. Is this no longer true? E.g., they exclude them in minGPT: https://github.com/karpathy/minGPT/blob/37baab71b9abea1b76ab...
    comment by zaptrem · 2025-10-05 · in What GPT-OSS leaks about OpenAI's training data
  3. …to deal with that. That said I strongly suggest everyone follow along with a GPT2 tutorial like: https://github.com/karpathy/minGPT to get a feel for what the big boys have to deal with. Any $100,000 training run can end in failure for no…
    comment by noosphr · 2025-03-24 · in Bitter Lesson is about AI agents
  4. nanoGPT contains some new ideas. https://github.com/karpathy/minGPT is more plain
    comment by Nimitz14 · 2025-01-29 · in SmolGPT: A minimal PyTorch implementation for training a small LLM from scratch
  5. …Transformer visualization: https://bbycroft.net/llm minGPT (Karpathy): https://github.com/karpathy/minGPT --- Next, some foundational textbooks for general ML and deep learning: Elements of Statistical Learning (aka the bible): h
    comment by xianshou · 2023-12-14 · in Ask HN: Daily practices for building AI/ML skills?
  6. …to 48? That also seems to correspond to the Channel size C referenced in the explanation text? https://github.com/karpathy/minGPT/blob/master/mingpt/model....
    comment by riemannzeta · 2023-12-03 · in LLM Visualization
  7. …it will be great to mention in the article since people without too much context might lose it [1] https://github.com/karpathy/minGPT/blob/master/mingpt/model....
    comment by warkanlock · 2023-12-03 · in LLM Visualization
  8. https://github.com/karpathy/minGPT
    comment by supriyo-biswas · 2023-11-24 · in Learn Machine Learning
  9. …for Facebook's original use case. We ran it on such a dataset and found out that directly using https://github.com/karpathy/minGPT consistently gives a better result. So we ended up using the output of Prophet as an input feature to a…
    comment by crvdgc · 2023-09-27 · in Prophet: Automatic Forecasting Procedure
  10. …slowdowns, what you're seeing is literally what it's spitting out in real time Here's an example https://github.com/karpathy/minGPT/blob/master/mingpt/model.... [1] Some things can be cached such as kv, but still
    comment by redox99 · 2023-02-17 · in We Found an Neuron in GPT-2
  11. Karpathy has a bunch of great resources on this front! His minGPT writeup is excellent https://github.com/karpathy/minGPT His more recent project nanoGPT which references this video is a much more capable, but still learning friendly,…
    comment by azath92 · 2023-02-09 · in A GPT in 60 Lines of NumPy
  12. …he is the person that's made a lot of cool things recently, including his own minimal GPT ( https://github.com/karpathy/minGPT ), and the much smaller version of it ( https://github.com/karpathy/nanoGPT ). But of course, since we are in…
    comment by tysam_and · 2023-02-09 · in A GPT in 60 Lines of NumPy
  13. …exactly, if there would be more to it I think he would be the kind of person to point it out. [0] https://github.com/karpathy/minGPT/tree/master/mingpt
    comment by quonn · 2022-12-11 · in The GPT Architecture, on a Napkin
  14. A small, very clearly written and well commented implementation: https://github.com/karpathy/minGPT
    comment by macleginn · 2022-12-11 · in The GPT Architecture, on a Napkin
  15. He contributed some commits to his excellent https://github.com/karpathy/minGPT during that time. BTW, Andrej, if you're reading this, it is not just excellent it is beyond excellent. I do a lot of tinkering with transformers and other…
    comment by lostmsu · 2022-07-14 · in Andrej Karpathy leaves Tesla
  16. …someone at Anthropic [4]. [1] https://www.oreilly.com/library/view/natural-language-proces... [2] https://github.com/karpathy/minGPT [3] https://jalammar.github.io/illustrated-transformer/ [4] https://blog.nelhage.com/post/transformers-for-…
    comment by moyix · 2022-04-16 · in The Principles of Deep Learning Theory
  17. Here's another simple implementation: https://github.com/karpathy/minGPT
    comment by p1esk · 2022-02-01 · in A Simple Transformer Implementation(PyTorch) Without Difficult Syntax
  18. …) And here's a super simple implementation of GPT by Andrej Karpathy. [link]( https://github.com/karpathy/minGPT/blob/master/mingpt/model.... )
    comment by silveraxe93 · 2020-10-26 · in Rethinking Attention with Performers