Hacker GitHub

← All repos

srush/llama2.rs

View on GitHub ↗ ★ 1,062 Rust

A fast llama2 decoder in pure Rust.

3 mentions on Hacker News by 3 people: 1 story, 2 comments · first 2023-08-05 · latest 2023-08-12

Mentions per year 3 mentions · busiest year 2023 (3)
20082026
AllStoriesComments NewestTopOldest
  1. This code runs Llama2 quantized and unquantized in a roughly minimal way: https://github.com/srush/llama2.rs (though extracting the quantized 70B weights takes a lot of RAM). I'm running the 13B quantized model on ~10-11GB of CPU memory.
    comment by singhrac · 2023-08-12 · in Ask HN: Cheapest hardware to run Llama 2 70B
  2. …candle or ggml, but just released a 4-bit rust llama2 implementation with simd. Runs pretty fast. https://github.com/srush/llama2.rs/
    comment by srush · 2023-08-08 · in Candle: Torch Replacement in Rust
  3. ▲ 60
    Llama2.rs: One-file Rust implementation of Llama2
    story by sshroot · 2023-08-05 · 10 comments