rasbt/LLMs-from-scratch
View on GitHub ↗ ★ 107,687 Jupyter Notebook
Implement a ChatGPT-like LLM in PyTorch from scratch, step by step
33 mentions on Hacker News by 22 people: 18 stories, 15 comments · first 2024-01-27 · latest 2026-10-26
20082026
-
…a Technical Book". He recommends reading offline and code-free first (although chapter-by-chapter). https://github.com/rasbt/LLMs-from-scratch/blob/cdbd33e6d57a...
comment by dtf · 2026-10-26 · in Sebastian Raschka's from-scratch video course on building reasoning models -
…is going to get you miles ahead of people who don't. For anyone who wants to dork around there is https://github.com/rasbt/LLMs-from-scratch which is something amazing that I think anyone who wants to engineer things around LLMs should at…
comment by boredumb · 2026-08-24 · in I were 17, I'd learn how to build LLMs from scratch -
▲ 2
DeepSeek Sparse Attentionstory by eigenBasis · 2026-05-24 · 0 comments
-
…[0][1][2]. Maybe it is a good problem to have to have to decide which learning resource to use. [0] https://github.com/rasbt/LLMs-from-scratch [1] https://www.manning.com/books/build-a-large-language-model-f... [2]…
comment by JoeDaDude · 2026-05-05 · in Train Your Own LLM from Scratch -
…Still a fun project, even if oversold. [1] https://spreadsheets-are-all-you-need.ai/ [2] https://github.com/rasbt/LLMs-from-scratch
comment by Propelloni · 2026-04-06 · in Show HN: I built a tiny LLM to demystify how language models work -
…that I would highly recommend Sebastian Raschka's book and youtube videos as way into the subject: https://github.com/rasbt/LLMs-from-scratch . Has anyone read TFA author Jay Alammar's book (published Oct 2024) and would they recommend it…
comment by libraryofbabel · 2025-12-22 · in The Illustrated Transformer -
▲ 1
Olmo 3 from Scratch (Standalone Notebook)story by quietlearning · 2025-11-24 · 0 comments
-
▲ 3
Explanation of Gated DeltaNet (Qwen3-Next and Kimi Linear)story by ModelForge · 2025-11-03 · 0 comments
-
Here is the code used in the book - https://github.com/rasbt/LLMs-from-scratch
comment by malshe · 2025-09-06 · in The maths you need to start understanding LLMs -
…larger Qwen3 0.6B model the A100 is faster (you can see it when scrolling to the bottom here: https://github.com/rasbt/LLMs-from-scratch/tree/main/ch05/11... )
comment by ModelForge · 2025-08-20 · in Gemma 3 270M re-implemented in pure PyTorch for local tinkering -
▲ 417
Gemma 3 270M re-implemented in pure PyTorch for local tinkeringstory by ModelForge · 2025-08-20 · 57 comments
-
▲ 3
Qwen3 from Scratchstory by tzury · 2025-08-02 · 0 comments
-
…game's response to user inputs but consistency in updating internal game logic. edit: found this! https://github.com/rasbt/LLMs-from-scratch/issues/249 Seems that it's an ongoing issue from various other links I've found, and now when I…
comment by lufenialif2 · 2025-07-02 · in LLMs Bring New Nature of Abstraction -
▲ 3
Qwen3 Implemented from Scratchstory by sbbq · 2025-06-20 · 0 comments
-
▲ 2
Implement a ChatGPT-like LLM in PyTorch from scratch, step by stepstory by jinqueeny · 2025-04-24 · 0 comments
-
▲ 1
Day 1 of trying to fit a Chatbot into a QR Codestory by kuberwastaken · 2025-04-20 · 2 comments
-
…the theory and how to actually implement one? The distinction between this and something like https://github.com/rasbt/LLMs-from-scratch is stark. My hot take is, if you haven’t built one, you don’t actually understand how they work, you…
comment by noodletheworld · 2025-03-06 · in Simple Explanation of LLMs -
PyTorch The book's code repo: https://github.com/rasbt/LLMs-from-scratch
comment by OnACoffeeBreak · 2025-03-05 · in Writing an LLM from scratch, part 8 – trainable self-attention -
▲ 2
Python Setup Tipsstory by tosh · 2025-02-16 · 0 comments
-
…with and without torch.compile (tldr it's 3x slower than a standard MHA when not compiled): https://github.com/rasbt/LLMs-from-scratch/blob/76e9a9ec02a1... EDIT: after taking a look at the repo, the only thing changed in the "46 commits…
comment by Scene_Cast2 · 2024-10-17 · in Triton Fork for Windows Support -
▲ 5
Implementing the Llama 3.2 1B and 3B Architectures from Scratchstory by sbbq · 2024-10-05 · 0 comments
-
▲ 2
Converting GPT to Llama step-by-step code guidestory by sbbq · 2024-09-23 · 0 comments
-
…LLM's do not possess the ability to perform the process detailed above. This is why it matters. 0 - https://github.com/rasbt/LLMs-from-scratch 1 - https://en.wikipedia.org/wiki/Large_language_model
comment by AdieuToLogic · 2024-09-02 · in Inductive or deductive? Rethinking the fundamental reasoning abilities of LLMs -
You can try this. https://github.com/rasbt/LLMs-from-scratch
comment by zacky31 · 2024-07-30 · in Ask HN: What are the best resources today for learning AI/LLMs -
▲ 2
LLM instruction finetuning from-scratch tutorialstory by sbbq · 2024-06-17 · 0 comments
-
▲ 2
Evaluating LLMs locally, on a laptop, with Llama 3 and Ollamastory by rasbt · 2024-06-13 · 0 comments
-
▲ 14
Finetuning an LLM-Based Spam Classifier with LoRA from Scratchstory by rasbt · 2024-05-11 · 0 comments
-
▲ 3
Finetune a GPT Model for Spam Detection on Your Laptop in Just 5 Minutesstory by rasbt · 2024-05-03 · 0 comments
-
▲ 2
Insights from Finetuning LLMs for Classification Tasksstory by rasbt · 2024-04-28 · 0 comments
-
…https://news.ycombinator.com/item?id=39156778 you can view the github for the book as they work on it https://github.com/rasbt/LLMs-from-scratch
comment by sandwichukulele · 2024-03-24 · in Ask HN: Textbook Regarding LLMs -
▲ 3
Comparing 5 ways to implement Multihead Attention in PyTorchstory by rasbt · 2024-03-08 · 0 comments
-
…be online in the MEAP soon, in the meantime the code along with the notes is also available here: https://github.com/rasbt/LLMs-from-scratch/blob/main/ch03/01... )
comment by rasbt · 2024-01-28 · in Implementing a ChatGPT-like LLM from scratch, step by step -
▲ 739
Implementing a ChatGPT-like LLM from scratch, step by stepstory by rasbt · 2024-01-27 · 98 comments