Hacker GitHub

← All repos

gmorenz/llama

View on GitHub ↗ ★ 35 Python

Inference code for LLaMA models

3 mentions on Hacker News by 1 person: 0 stories, 3 comments · first 2023-03-08 · latest 2023-03-13

Mentions per year 3 mentions · busiest year 2023 (3)
20082026
AllStoriesComments NewestTopOldest
  1. …GPU pretty efficiently. If you have >= 32GB ram and >=2GB vram my code here should work for you: https://github.com/gmorenz/llama/tree/gpu_offload There's probably a cleaner version of it somewhere else. Really you should only need >= 16…
    comment by gpm · 2023-03-13 · in Alpaca: A strong open-source instruction-following model
  2. My code for this is very much not high quality, but I have a CPU + GPU + SSD combination: https://github.com/gmorenz/llama/tree/ssd Usage instructions in the commit message: https://github.com/facebookresearch/llama/commit/5be06e56056...…
    comment by gpm · 2023-03-11 · in Llama.cpp: Port of Facebook's LLaMA model in C/C++, with Apple Silicon support
  3. …still be the bottleneck) A lack of an absurd number of CPUs just means it's slow, not impossible. https://github.com/gmorenz/llama/tree/ssd
    comment by gpm · 2023-03-08 · in Fork of Facebook’s LLaMa model to run on CPU