Hacker GitHub

← All repos

mozilla-ai/llamafile

View on GitHub ↗ ★ 26,545 C++

Distribute and run LLMs with a single file.

112 mentions on Hacker News by 82 people: 10 stories, 102 comments · first 2023-11-29 · latest 2026-10-27

Mentions per year 112 mentions · busiest year 2024 (62)
20082026
AllStoriesComments NewestTopOldest
  1. I'm assuming you're on Apple Silicon? Please follow https://github.com/Mozilla-Ocho/llamafile/issues/11 which is tracking this. We've received multiple reports even though I personally haven't figured out how to reproduce it yet.
    comment by jart · 2023-11-30 · in Llamafile lets you distribute and run LLMs with a single file
  2. …and pass any GGUF model as an argument. > llamafile-server-0.1 -m llama-2-13b.Q8_0.gguf https://github.com/Mozilla-Ocho/llamafile/releases/tag/0.1
    comment by column · 2023-11-30 · in Llamafile lets you distribute and run LLMs with a single file
  3. …llama.cpp project upstream that you'll need in order to run these weights using llamafile. Enjoy! https://github.com/Mozilla-Ocho/llamafile/commit/865462fc465...
    comment by jart · 2023-11-30 · in Llamafile lets you distribute and run LLMs with a single file
  4. Sounds like you should download the 4.45MB llamafile-server-0.1 executable from https://github.com/Mozilla-Ocho/llamafile/releases/tag/0.1 and then run it against your existing gguf model files like this: ./llamafile-server-0.1 -m…
    comment by simonw · 2023-11-30 · in Llamafile lets you distribute and run LLMs with a single file
  5. …Please see the Gotchas section of the README for the copy/pastable commands you can run: https://github.com/mozilla-Ocho/llamafile#gotchas
    comment by jart · 2023-11-30 · in Llamafile lets you distribute and run LLMs with a single file
  6. …I'm not aware of any sh that still doesn't support APE. See the Gotchas section of the README https://github.com/mozilla-Ocho/llamafile#gotchas
    comment by jart · 2023-11-30 · in Llamafile lets you distribute and run LLMs with a single file
  7. You'd do something like this on PowerShell: curl -Lo llamafile.exe https://github.com/Mozilla-Ocho/llamafile/releases/download/0.1/llamafile-server-0.1 curl -Lo wizard.gguf https://huggingface.co/TheBloke/WizardCoder-Python-13B-V1.0-GGUF/re…
    comment by jart · 2023-11-30 · in Llamafile lets you distribute and run LLMs with a single file
  8. …tasks - the file should be everything you need. There are some gotchas to watch out for though: https://github.com/mozilla-Ocho/llamafile#gotchas
    comment by simonw · 2023-11-29 · in Llamafile lets you distribute and run LLMs with a single file
  9. …is 0xfe1c0ed4 bytes in size, which is just 30MB shy of that limit. https://github.com/Mozilla-Ocho/llamafile/commit/81c6ad3251f...
    comment by jart · 2023-11-29 · in Llamafile lets you distribute and run LLMs with a single file
  10. The binaries themselves are available standalone https://github.com/Mozilla-Ocho/llamafile/releases
    comment by stevenhuang · 2023-11-29 · in Llamafile lets you distribute and run LLMs with a single file
  11. The technical details in the README are quite an interesting read: https://github.com/mozilla-Ocho/llamafile#technical-details
    comment by polyrand · 2023-11-29 · in Llamafile lets you distribute and run LLMs with a single file
  12. ▲ 1,075
    Llamafile lets you distribute and run LLMs with a single file
    story by tfinch · 2023-11-29 · 288 comments