Hacker GitHub

← All repos

tysam-code/hlb-CIFAR10

View on GitHub ↗ ★ 1,317 Python

Train to 94% on CIFAR-10 in <6.3 seconds on a single A100. Or ~95.79% in ~110 seconds (or less!)

23 mentions on Hacker News by 8 people: 4 stories, 19 comments · first 2022-12-12 · latest 2025-09-06

Mentions per year 23 mentions · busiest year 2023 (16)
20082026
AllStoriesComments NewestTopOldest
  1. …benchmarks, nearly tripling the speed of a longstanding CIFAR10 world record (~17.1s -> 6.3s, https://github.com/tysam-code/hlb-CIFAR10 , recognized by Karpathy: https://x.com/karpathy/status/1620103412686942208 ), setting a modded-nanogpt…
    comment by fern_ · 2025-09-06 · in Ask HN: Who wants to be hired? (September 2025)
  2. …a few benchmarks, nearly tripling the speed of a longstanding CIFAR10 world record (~17.1s -> 6.3s, https://github.com/tysam-code/hlb-CIFAR10 , recognized by Karpathy: https://x.com/karpathy/status/1620103412686942208 ), setting a…
    comment by fern_ · 2025-08-01 · in Ask HN: Who wants to be hired? (August 2025)
  3. Hah why is this on HN today? Update from 2024: https://github.com/tysam-code/hlb-CIFAR10 Train to 94% on CIFAR-10 in <6.3 seconds on a single A100. Or ~95.79% in ~110 seconds (or less!) https://paperswithcode.com/sota/image-classification-o…
    comment by karpathy · 2024-04-10 · in Lessons learned from manually classifying CIFAR-10 (2011)
  4. A training speed project building on https://github.com/tysam-code/hlb-CIFAR10 to reach faster times
    comment by kjjnot · 2024-04-04 · in Train to 94% on CIFAR-10 in 3.29 seconds on a single A100
  5. …Crazy optimizations (no affiliation): 94% on CIFAR-10 in <6.3 seconds on a single A100 : https://github.com/tysam-code/hlb-CIFAR10 I also want to give maybe some better information about ViTs in general. Lucas Beyer is a good source and…
    comment by stevenwalton · 2023-12-02 · in Deep Dive into the Vision Transformers Paper
  6. …accuracy in <13.6 seconds on a V100, est. < 6.5 seconds on an A100. Made to be edited/run in Colab: https://github.com/tysam-code/hlb-CIFAR10 It's originally kitted for CIFAR10, but I've found the parameters to be quite general. The code…
    comment by tysam_and · 2023-09-20 · in 78% MNIST accuracy using GZIP in under 10 lines of code
  7. Sure. Basically everything in https://github.com/tysam-code/hlb-CIFAR10 was directly founded on concepts shared in the above paper, down to the coding, commenting, and layout styles (hence why I advocate so strongly for it as a requirement…
    comment by tysam_and · 2023-08-16 · in The Mathematics of Training LLMs
  8. …I do have a bit of an actual minor personal beef with him. He said he was going to reimplement https://github.com/tysam-code/hlb-CIFAR10 in tinygrad, bashed a few parts of the code (my feelings!) for a while on stream, and then gave up a…
    comment by tysam_and · 2023-08-11 · in There is no hard takeoff
  9. You can control range with a temperature, works pretty well! https://github.com/tysam-code/hlb-CIFAR10/blob/3bb104ce16d16...
    comment by tysam_and · 2023-05-23 · in In Defense of Pure 16-Bit Floating-Point Neural Networks
  10. …to take a look (I'll note that bf16 is simpler on the whole for a few reasons, generally seamless): https://github.com/tysam-code/hlb-CIFAR10 [for fp16] and https://github.com/tysam-code/hlb-gpt [for bf16] Personally from my experience, I…
    comment by tysam_and · 2023-05-23 · in In Defense of Pure 16-Bit Floating-Point Neural Networks
  11. …tiny neural network (<10 MB if just saved raw out to disk as a definition file). That's located at https://github.com/tysam-code/hlb-CIFAR10 . You could make that even smaller if you wanted to, though at least this network is already…
    comment by tysam_and · 2023-05-22 · in Neural Network Architecture Beyond Width and Depth
  12. …better than symbols, this could be a helpful adjunct as well if you're wanting to dig deep a bit. https://github.com/tysam-code/hlb-CIFAR10 (convolutional networks and imaging stuff. it's fast. very fast.)…
    comment by tysam_and · 2023-04-02 · in MIT 6.S191: Recurrent Neural Networks, Transformers, and Attention [video]
  13. …of it, but it is a fun explanation). In fact (though this repo performs several functions), I made https://github.com/tysam-code/hlb-CIFAR10 as beginner-friendly as possible. One does have to make some decisions to keep verbosity down, and…
    comment by tysam_and · 2023-02-09 · in A GPT in 60 Lines of NumPy
  14. ….85 * 4.8e-4 * batchsize/bias_scaler` instead of `'bias_decay': 4.08e-4 * batchsize/bias_scaler`? https://github.com/tysam-code/hlb-CIFAR10/blob/84ced1ac1dc70...
    comment by rsfern · 2023-01-30 · in Train CIFAR10 to 94% in under 10 seconds on a single A100
  15. I was wondering, what are these magic numbers on line 84[1]? [1] https://github.com/tysam-code/hlb-CIFAR10/blob/main/main.py#...
    comment by newswasboring · 2023-01-30 · in Train CIFAR10 to 94% in under 10 seconds on a single A100
  16. …fiddly ones of yore. Anywho -- here's what I ran into and all of the requisite debugging with it: https://github.com/tysam-code/hlb-CIFAR10/issues/2#issuecomm... Currently, MaxPooling is one of the slowest operations per-byte of memory…
    comment by tysam_and · 2023-01-30 · in Train CIFAR10 to 94% in under 10 seconds on a single A100
  17. …the minGPT treatment on the current best single-GPU Dawnbench entry, added a few tweaks and got https://github.com/tysam-code/hlb-CIFAR10 . It currently (AFAIK) holds the world record for training to the 94% mark by a fair bit. It's about…
    comment by tysam_and · 2023-01-18 · in Let's build GPT: from scratch, in code, spelled out by Andrej Karpathy [video]
  18. …read through the direct code here if you're willing to wade through some of the mathy code on it: https://github.com/tysam-code/hlb-CIFAR10/blob/main/main.py#...
    comment by tbalsam · 2022-12-23 · in Do simpler machine learning models exist and how can we find them?
  19. …downsides that come with that. If you're interested or have any questions, let me know! :D :)))) https://github.com/tysam-code/hlb-CIFAR10
    comment by tbalsam · 2022-12-22 · in Do simpler machine learning models exist and how can we find them?