Hacker GitHub

← All repos

vllm-project/llm-compressor

View on GitHub ↗ ★ 3,957 Python

State-of-the-art LLM compression, built for production inference with vLLM

5 mentions on Hacker News by 3 people: 1 story, 4 comments · first 2024-08-20 · latest 2026-08-19

Mentions per year 5 mentions · busiest year 2026 (3)
20082026
AllStoriesComments NewestTopOldest
  1. …hardware that does not fit the model, by processing the weights layer by layer. I use this project: https://github.com/vllm-project/llm-compressor
    comment by verdverm · 2026-08-19 · in Unsloth Dynamic 3.0 GGUFs
  2. …using, llm-compressor, can quant models that do not fit in memory (use the sequential pipeline) https://github.com/vllm-project/llm-compressor my setup to help you on your way: https://github.com/verdverm/quantr Though it seems these will…
    comment by verdverm · 2026-07-21 · in Laguna S 2.1
  3. …have to wait for open source maintainers to get back to me about a bug to keep making progress. https://github.com/vllm-project/llm-compressor/issues/2914
    comment by verdverm · 2026-07-12 · in LLMs are still just low code / no code software
  4. …a one-time cost). But if you land on a Q4 model, it's not a bad choice once you have a favorite. 1: https://github.com/vllm-project/llm-compressor
    comment by reissbaker · 2025-05-30 · in Ask HN: What is the best LLM for consumer grade hardware?