Hacker GitHub

← All repos

bitsandbytes-foundation/bitsandbytes

View on GitHub ↗ ★ 8,535 Python

Accessible large language models via k-bit quantization for PyTorch.

9 mentions on Hacker News by 8 people: 2 stories, 7 comments · first 2023-04-05 · latest 2024-06-25

Mentions per year 9 mentions · busiest year 2023 (7)
20082026
AllStoriesComments NewestTopOldest
  1. …dependency in LLM world) there's a ROCm fork that the AMD maintainers are trying to merge in ( https://github.com/TimDettmers/bitsandbytes/issues/107 ). Meanwhile an Intel employee merged a change that made a common device abstraction…
    comment by singhrac · 2024-06-25 · in Testing AMD's Giant MI300X
  2. Found it from https://github.com/Vahe1994/SpQR Was somehow expecting it to be at https://github.com/TimDettmers/bitsandbytes . My bad.
    comment by AhtiK · 2024-03-08 · in Fine tune a 70B language model at home
  3. …[2]: https://huggingface.co/DiscoResearch/mixtral-7b-8expert [3]: https://github.com/TimDettmers/bitsandbytes/tree/sparse_moe
    comment by espadrine · 2023-12-09 · in French AI startup Mistral secures €2B valuation
  4. ▲ 15
    Ask HN: Are you training and running custom LLMs and how are you doing it?
    story by kordlessagain · 2023-08-14 · 2 comments
  5. …torch/pytorch/issues?q=is%3Aissue+is%3A... [2] https://github.com/microsoft/DeepSpeed/issues/1580 [3] https://github.com/TimDettmers/bitsandbytes/issues/485 [4] https://github.com/AUTOMATIC1111/stable-diffusion-webui/disc... [5]…
    comment by lhl · 2023-07-26 · in Guide to running Llama 2 locally
  6. Release notes: https://github.com/TimDettmers/bitsandbytes/releases/tag/0.4...
    comment by convexstrictly · 2023-07-11 · in Model 4 bit inference 4.2x faster than 16 bit with full HF support
  7. ▲ 1
    Bitsandbytes: 8-bit CUDA functions for PyTorch
    story by anotherpaulg · 2023-05-25 · 0 comments
  8. …more to make low-resource LLMs usable than anyone else. First bitsandbytes[1] and now this. [1] https://github.com/TimDettmers/bitsandbytes
    comment by nl · 2023-05-24 · in QLoRA: Efficient Finetuning of Quantized LLMs
  9. …128` although there have been reports that bitsandbytes have problems w/ 4bit perf on some cards: https://github.com/TimDettmers/bitsandbytes/issues/181
    comment by lhl · 2023-04-05 · in State-of-the-art open-source chatbot, Vicuna-13B, just released model weights