rasbt/LLMs-from-scratch
View on GitHub ↗ ★ 107,687 Jupyter Notebook
Implement a ChatGPT-like LLM in PyTorch from scratch, step by step
33 mentions on Hacker News by 22 people: 18 stories, 15 comments · first 2024-01-27 · latest 2026-10-26
20082026
-
…a Technical Book". He recommends reading offline and code-free first (although chapter-by-chapter). https://github.com/rasbt/LLMs-from-scratch/blob/cdbd33e6d57a...
comment by dtf · 2026-10-26 · in Sebastian Raschka's from-scratch video course on building reasoning models -
…is going to get you miles ahead of people who don't. For anyone who wants to dork around there is https://github.com/rasbt/LLMs-from-scratch which is something amazing that I think anyone who wants to engineer things around LLMs should at…
comment by boredumb · 2026-08-24 · in I were 17, I'd learn how to build LLMs from scratch -
…[0][1][2]. Maybe it is a good problem to have to have to decide which learning resource to use. [0] https://github.com/rasbt/LLMs-from-scratch [1] https://www.manning.com/books/build-a-large-language-model-f... [2]…
comment by JoeDaDude · 2026-05-05 · in Train Your Own LLM from Scratch -
…Still a fun project, even if oversold. [1] https://spreadsheets-are-all-you-need.ai/ [2] https://github.com/rasbt/LLMs-from-scratch
comment by Propelloni · 2026-04-06 · in Show HN: I built a tiny LLM to demystify how language models work -
…that I would highly recommend Sebastian Raschka's book and youtube videos as way into the subject: https://github.com/rasbt/LLMs-from-scratch . Has anyone read TFA author Jay Alammar's book (published Oct 2024) and would they recommend it…
comment by libraryofbabel · 2025-12-22 · in The Illustrated Transformer -
Here is the code used in the book - https://github.com/rasbt/LLMs-from-scratch
comment by malshe · 2025-09-06 · in The maths you need to start understanding LLMs -
…larger Qwen3 0.6B model the A100 is faster (you can see it when scrolling to the bottom here: https://github.com/rasbt/LLMs-from-scratch/tree/main/ch05/11... )
comment by ModelForge · 2025-08-20 · in Gemma 3 270M re-implemented in pure PyTorch for local tinkering -
…game's response to user inputs but consistency in updating internal game logic. edit: found this! https://github.com/rasbt/LLMs-from-scratch/issues/249 Seems that it's an ongoing issue from various other links I've found, and now when I…
comment by lufenialif2 · 2025-07-02 · in LLMs Bring New Nature of Abstraction -
…the theory and how to actually implement one? The distinction between this and something like https://github.com/rasbt/LLMs-from-scratch is stark. My hot take is, if you haven’t built one, you don’t actually understand how they work, you…
comment by noodletheworld · 2025-03-06 · in Simple Explanation of LLMs -
PyTorch The book's code repo: https://github.com/rasbt/LLMs-from-scratch
comment by OnACoffeeBreak · 2025-03-05 · in Writing an LLM from scratch, part 8 – trainable self-attention -
…with and without torch.compile (tldr it's 3x slower than a standard MHA when not compiled): https://github.com/rasbt/LLMs-from-scratch/blob/76e9a9ec02a1... EDIT: after taking a look at the repo, the only thing changed in the "46 commits…
comment by Scene_Cast2 · 2024-10-17 · in Triton Fork for Windows Support -
…LLM's do not possess the ability to perform the process detailed above. This is why it matters. 0 - https://github.com/rasbt/LLMs-from-scratch 1 - https://en.wikipedia.org/wiki/Large_language_model
comment by AdieuToLogic · 2024-09-02 · in Inductive or deductive? Rethinking the fundamental reasoning abilities of LLMs -
You can try this. https://github.com/rasbt/LLMs-from-scratch
comment by zacky31 · 2024-07-30 · in Ask HN: What are the best resources today for learning AI/LLMs -
…https://news.ycombinator.com/item?id=39156778 you can view the github for the book as they work on it https://github.com/rasbt/LLMs-from-scratch
comment by sandwichukulele · 2024-03-24 · in Ask HN: Textbook Regarding LLMs -
…be online in the MEAP soon, in the meantime the code along with the notes is also available here: https://github.com/rasbt/LLMs-from-scratch/blob/main/ch03/01... )
comment by rasbt · 2024-01-28 · in Implementing a ChatGPT-like LLM from scratch, step by step