Hacker GitHub

← All repos

rustformers/llm

View on GitHub ↗ ★ 6,150 Rust archived

[Unmaintained, see README] An ecosystem of Rust libraries for working with large language models

9 mentions on Hacker News by 8 people: 3 stories, 6 comments · first 2023-03-14 · latest 2023-04-20

Mentions per year 9 mentions · busiest year 2023 (9)
20082026
AllStoriesComments NewestTopOldest
  1. https://github.com/rustformers/llama-rs
    comment by MacsHeadroom · 2023-04-20 · in StableLM: A new open-source language model
  2. Hi! I'm a maintainer of https://github.com/rustformers/llama-rs . We're planning to expand our model support soon.
    comment by Philpax · 2023-04-19 · in StableLM: A new open-source language model
  3. ▲ 5
    Run LLaMA Inference on CPU, with Rust
    story by kristianpaul · 2023-04-05 · 0 comments
  4. I'm curious if someone will have to port these enhancements elsewhere, ie: https://github.com/rustformers/llama-rs
    comment by solarpunk · 2023-04-01 · in Llama.cpp 30B runs with only 6GB of RAM now
  5. …one discussion item has been to run a verification and common benchmark for the implementations. https://github.com/setzer22/llama-rs/issues/4 I've mostly heard that, at least for the larger models, quantization has barely any noticeable…
    comment by adeon · 2023-03-16 · in Show HN: Alpaca.cpp – Run an Instruction-Tuned Chat-Style LLM on a MacBook
  6. …implementations on r/rust subreddit this week: https://github.com/Noeda/rllama/ (pure Rust+OpenCL) https://github.com/setzer22/llama-rs/ (ggml based) https://github.com/philpax/ggllama (also ggml based) There's also a discussion on GitHub…
    comment by adeon · 2023-03-15 · in Llama.rs – Rust port of llama.cpp for fast LLaMA inference on CPU
  7. …happen and try it out. Looking at how the inference runs, this shouldn't be a big problem, right? https://github.com/setzer22/llama-rs/blob/main/llama-rs/src/...
    comment by ilovefood · 2023-03-15 · in Llama.rs – Rust port of llama.cpp for fast LLaMA inference on CPU
  8. ▲ 202
    Llama.rs – Rust port of llama.cpp for fast LLaMA inference on CPU
    story by rrampage · 2023-03-15 · 24 comments
  9. ▲ 8
    LLaMA-rs: a Rust port of llama.cpp for fast LLaMA inference on CPU
    story by darthdeus · 2023-03-14 · 0 comments