mit-han-lab/llm-awq
View on GitHub ↗ ★ 3,664 Python
[MLSys 2024 Best Paper Award] AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration
3 mentions on Hacker News by 3 people: 1 story, 2 comments · first 2023-06-02 · latest 2023-12-08
20082026
-
…serving framework for 4-bit LLMs by AWQ. It delivers 2.3x generation speed up on RTX4090. Code: https://github.com/mit-han-lab/llm-awq/tree/main/tinychat
comment by enduku · 2023-12-08 · in TinyChat: Large Language Model on the Edge -
▲ 1
New hardware-friendly quantization methodstory by theolivenbaum · 2023-06-02 · 0 comments
-
…LLMs. The technique has already been applied to both Vicuna and the multimodal LLaMA variant LLaVA. https://github.com/mit-han-lab/llm-awq
comment by valine · 2023-06-02 · in Activation-Aware Weight Quantization for LLM Compression Outperforms GPTQ