bitsandbytes-foundation/bitsandbytes
View on GitHub ↗ ★ 8,535 Python
Accessible large language models via k-bit quantization for PyTorch.
9 mentions on Hacker News by 8 people: 2 stories, 7 comments · first 2023-04-05 · latest 2024-06-25
20082026
-
…dependency in LLM world) there's a ROCm fork that the AMD maintainers are trying to merge in ( https://github.com/TimDettmers/bitsandbytes/issues/107 ). Meanwhile an Intel employee merged a change that made a common device abstraction…
comment by singhrac · 2024-06-25 · in Testing AMD's Giant MI300X -
Found it from https://github.com/Vahe1994/SpQR Was somehow expecting it to be at https://github.com/TimDettmers/bitsandbytes . My bad.
comment by AhtiK · 2024-03-08 · in Fine tune a 70B language model at home -
…[2]: https://huggingface.co/DiscoResearch/mixtral-7b-8expert [3]: https://github.com/TimDettmers/bitsandbytes/tree/sparse_moe
comment by espadrine · 2023-12-09 · in French AI startup Mistral secures €2B valuation -
…torch/pytorch/issues?q=is%3Aissue+is%3A... [2] https://github.com/microsoft/DeepSpeed/issues/1580 [3] https://github.com/TimDettmers/bitsandbytes/issues/485 [4] https://github.com/AUTOMATIC1111/stable-diffusion-webui/disc... [5]…
comment by lhl · 2023-07-26 · in Guide to running Llama 2 locally -
Release notes: https://github.com/TimDettmers/bitsandbytes/releases/tag/0.4...
comment by convexstrictly · 2023-07-11 · in Model 4 bit inference 4.2x faster than 16 bit with full HF support -
…more to make low-resource LLMs usable than anyone else. First bitsandbytes[1] and now this. [1] https://github.com/TimDettmers/bitsandbytes
comment by nl · 2023-05-24 · in QLoRA: Efficient Finetuning of Quantized LLMs -
…128` although there have been reports that bitsandbytes have problems w/ 4bit perf on some cards: https://github.com/TimDettmers/bitsandbytes/issues/181
comment by lhl · 2023-04-05 · in State-of-the-art open-source chatbot, Vicuna-13B, just released model weights