Hacker GitHub

← All repos

samuel-vitorino/lm.rs

View on GitHub ↗ ★ 1,032 Rust

Minimal LLM inference in Rust

3 mentions on Hacker News by 2 people: 1 story, 2 comments · first 2024-10-11 · latest 2024-10-11

Mentions per year 3 mentions · busiest year 2024 (3)
20082026
AllStoriesComments NewestTopOldest
  1. …speedy and used 1000% of CPU across 13 threads (according to Activity Monitor). cd /tmp git clone https://github.com/samuel-vitorino/lm.rs cd lm.rs RUSTFLAGS="-C target-cpu=native" cargo build --release --bin chat curl -LO…
    comment by simonw · 2024-10-11 · in Lm.rs: Minimal CPU LLM inference in Rust with no dependency
  2. ▲ 310
    Lm.rs: Minimal CPU LLM inference in Rust with no dependency
    story by littlestymaar · 2024-10-11 · 76 comments
  3. …even PyTorch as dependency there's a straightforward CPU implementation of llama/gemma in Rust: https://github.com/samuel-vitorino/lm.rs/ It's impressive to realize how little code is needed to run these models at all.
    comment by littlestymaar · 2024-10-11 · in Run Llama locally with only PyTorch on CPU