microsoft/LLMLingua

View on GitHub ↗ ★ 6,816 Python

[EMNLP'23, ACL'24] To speed up LLMs' inference and enhance LLM's perceive of key information, compress the prompt and KV-Cache, which achieves up to 20x compression with minimal performance loss.

3 mentions on Hacker News by 3 people: 2 stories, 1 comment · first 2023-12-18 · latest 2025-08-26

Mentions per year 3 mentions · busiest year 2025 (2)

20082026

All Stories Comments Newest Top Oldest

…https://github.com/yamadashy/repomix 2. To reduce token, compress the file with LLMLingua-2: https://github.com/microsoft/LLMLingua (fewer token = more context can be given to LLM = LLM better understands your repository) 3. Copy & Paste…
comment by letaem77 · 2025-08-26 · in DeepWiki: Understand Any Codebase
▲ 2
Show HN: Tokiwi – An online tokenizer for any Hugging Face model
story by TweedBeetle · 2025-03-29 · 0 comments
▲ 149
LLMLingua: Compressing Prompts for Faster Inferencing
story by TarqDirtyToMe · 2023-12-18 · 47 comments