Hacker GitHub

← All repos

lyogavin/airllm

View on GitHub ↗ ★ 41,574 Jupyter Notebook

AirLLM 2.8T LLM inference with single 4GB GPU

9 mentions on Hacker News by 9 people: 4 stories, 5 comments · first 2024-08-01 · latest 2026-08-07

Mentions per year 9 mentions · busiest year 2026 (6)
20082026
AllStoriesComments NewestTopOldest
  1. ▲ 1
    Show HN: Kimi K3 inference on about 100k PS3 nodes
    story by anticensor · 2026-08-07 · 1 comment
  2. For anyone wondering “how slow is this?” IIUC, Kimi K3 on RTX 6000 Ada (48GB) takes 292 s/token https://github.com/lyogavin/airllm/releases/tag/v3.1.0
    comment by imenani · 2026-08-03 · in AirLLM 70B inference with single 4GB GPU
  3. ▲ 232
    AirLLM 70B inference with single 4GB GPU
    story by Anon84 · 2026-08-03 · 84 comments
  4. ▲ 2
    AirLLM: Inference 2.8T Kimi K3 on a single 4GB GPU
    story by maxloh · 2026-08-02 · 0 comments
  5. ▲ 2
    AirLLM optimizes inference memory usage
    story by nreece · 2026-03-08 · 0 comments
  6. …of SSDs will be significantly slower than RAM. That said, there are folks out there doing it. https://github.com/lyogavin/airllm is one example.
    comment by timschmidt · 2026-01-28 · in Trinity large: An open 400B sparse MoE model
  7. I have not used it but this is probably it: https://github.com/lyogavin/airllm
    comment by skirmish · 2025-09-12 · in Qwen3-Next
  8. …models. You can run the biggest models such as llama 3.1 405B on a fairly low end desktop today: https://github.com/lyogavin/airllm However, it will be far slower as you said.
    comment by ryao · 2024-11-20 · in Llama 3.1 405B now runs at 969 tokens/s on Cerebras Inference
  9. https://github.com/lyogavin/airllm What sort of speed do you get?
    comment by langcss · 2024-08-01 · in Show HN: Run Llama3.1 405B on a 8GB VRAM challenge [video]