Hacker GitHub

← All repos

rustformers/llm

View on GitHub ↗ ★ 6,150 Rust archived

[Unmaintained, see README] An ecosystem of Rust libraries for working with large language models

9 mentions on Hacker News by 8 people: 3 stories, 6 comments · first 2023-03-14 · latest 2023-04-20

Mentions per year 9 mentions · busiest year 2023 (9)
20082026
AllStoriesComments NewestTopOldest
  1. https://github.com/rustformers/llama-rs
    comment by MacsHeadroom · 2023-04-20 · in StableLM: A new open-source language model
  2. Hi! I'm a maintainer of https://github.com/rustformers/llama-rs . We're planning to expand our model support soon.
    comment by Philpax · 2023-04-19 · in StableLM: A new open-source language model
  3. I'm curious if someone will have to port these enhancements elsewhere, ie: https://github.com/rustformers/llama-rs
    comment by solarpunk · 2023-04-01 · in Llama.cpp 30B runs with only 6GB of RAM now
  4. …one discussion item has been to run a verification and common benchmark for the implementations. https://github.com/setzer22/llama-rs/issues/4 I've mostly heard that, at least for the larger models, quantization has barely any noticeable…
    comment by adeon · 2023-03-16 · in Show HN: Alpaca.cpp – Run an Instruction-Tuned Chat-Style LLM on a MacBook
  5. …implementations on r/rust subreddit this week: https://github.com/Noeda/rllama/ (pure Rust+OpenCL) https://github.com/setzer22/llama-rs/ (ggml based) https://github.com/philpax/ggllama (also ggml based) There's also a discussion on GitHub…
    comment by adeon · 2023-03-15 · in Llama.rs – Rust port of llama.cpp for fast LLaMA inference on CPU
  6. …happen and try it out. Looking at how the inference runs, this shouldn't be a big problem, right? https://github.com/setzer22/llama-rs/blob/main/llama-rs/src/...
    comment by ilovefood · 2023-03-15 · in Llama.rs – Rust port of llama.cpp for fast LLaMA inference on CPU