Hacker GitHub

← All repos

RWKV/rwkv.cpp

View on GitHub ↗ ★ 1,591 C++

INT4/INT5/INT8 and FP16 inference on CPU for RWKV language model

5 mentions on Hacker News by 5 people: 0 stories, 5 comments · first 2023-04-11 · latest 2024-01-29

Mentions per year 5 mentions · busiest year 2023 (4)
20082026
AllStoriesComments NewestTopOldest
  1. There's https://github.com/saharNooby/rwkv.cpp , which related-ish[0] to ggml/llama.cpp [0]: https://github.com/ggerganov/llama.cpp/issues/846
    comment by kristjansson · 2024-01-29 · in Eagle 7B: Soaring past Transformers
  2. …is applicable to RWKV as much as transformers. Most notably in our RWKV.cpp community project: https://github.com/saharNooby/rwkv.cpp Tooling/Ecosystem is something that I am actively working on as there is still a gap to transformers…
    comment by pico_creator · 2023-09-04 · in The Eleuther AI Mafia
  3. …algorithms of most LMM implementations (rwkv.cpp has a relatively clean implementation in python https://github.com/saharNooby/rwkv.cpp/blob/master/rwkv/samp... )
    comment by spion · 2023-07-21 · in Llama: Add grammar-based sampling
  4. …might see here how bigger models have much worse perplexity with 4bit due to the weight outliers: https://github.com/saharNooby/rwkv.cpp/issues/12 For LLaMA models - yeah, different story.
    comment by Ambix · 2023-05-18 · in Numbers every LLM developer should know
  5. Also worth checking out https://github.com/saharNooby/rwkv.cpp which is based on Georgi's library and offers support for the RWKV family of models which are Apache-2.0 licensed.
    comment by abetlen · 2023-04-11 · in The Coming of Local LLMs