Hacker GitHub

← All repos

mozilla-ai/llamafile

View on GitHub ↗ ★ 26,545 C++

Distribute and run LLMs with a single file.

112 mentions on Hacker News by 82 people: 10 stories, 102 comments · first 2023-11-29 · latest 2026-10-27

Mentions per year 112 mentions · busiest year 2024 (62)
20082026
AllStoriesComments NewestTopOldest
  1. Any performance benchmark against 'llamafile'[0] or others? [0] - https://github.com/mozilla-Ocho/llamafile
    comment by DrNosferatu · 2024-04-03 · in PyTorch Library for Running LLM on Intel CPU and GPU
  2. Ollama https://ollama.com/ or llamafile https://github.com/Mozilla-Ocho/llamafile *there are some tools/projects already integrating it that way
    comment by throwaway888abc · 2024-04-01 · in Ask HN: CPU-based LLM for Markdown editor
  3. …sudo sh -c "echo ':APE-jart:M::jartsr::/usr/bin/ape:' >/proc/sys/fs/binfmt_misc/register" https://github.com/mozilla-ocho/llamafile/?tab=readme-ov-fil...
    comment by jart · 2024-04-01 · in LLaMA now goes faster on CPUs
  4. …a binary with a few command-line options, seems pretty close to what you describe you want. [1] https://github.com/Mozilla-Ocho/llamafile
    comment by TachyonicBytes · 2024-04-01 · in Ask HN: What is the current (Apr. 2024) gold standard of running an LLM locally?
  5. …the README "gotcha" section). Solved my running "assimilate" [3] on the downloaded llamafile. [1] https://github.com/Mozilla-Ocho/llamafile/ [2] https://huggingface.co/jartine/Mixtral-8x7B-Instruct-v0.1-llamafile/resolve/main/mixtral-8x7b-i…
    comment by yaantc · 2024-04-01 · in LLaMA now goes faster on CPUs
  6. …will also benefit. Though of course everyone should also look directly at releases from llamafile https://github.com/mozilla-Ocho/llamafile .
    comment by dagaci · 2024-04-01 · in LLaMA now goes faster on CPUs
  7. …also happen to be actually portable executables, ehm, excuse me, αcτµαlly pδrταblε εxεcµταblες. https://github.com/Mozilla-Ocho/llamafile?tab=readme-ov-file...
    comment by formerly_proven · 2024-03-31 · in Llamafile 0.7 Brings AVX-512 Support: 10x Faster Prompt Eval Times for AMD Zen 4
  8. …file size limit. For Windows users, here's an example for the Mistral LLM: curl -L -o llamafile.exe https://github.com/Mozilla-Ocho/llamafile/releases/download/... curl -L -o mistral.gguf https://huggingface.co/TheBloke/Mistral-7B-Instruct-…
    comment by supposemaybe · 2024-03-31 · in Llamafile 0.7 Brings AVX-512 Support: 10x Faster Prompt Eval Times for AMD Zen 4
  9. …giving credit and can continue having a nice single file binary. The code which does that is here: https://github.com/Mozilla-Ocho/llamafile/blob/9c53c272650bb... So let me install the latest Ollama on my Windows computer and see if…
    comment by jart · 2024-03-16 · in Ollama now supports AMD graphics cards
  10. …don't avoid it altogether, and they certainly don't seem to think its unnecessary. (source: https://github.com/Mozilla-Ocho/llamafile ) - 2nd sentence in README: "Our goal is to make open LLMs much more accessible to both developers and…
    comment by refulgentis · 2024-03-16 · in Ollama now supports AMD graphics cards
  11. …would be cool to also discuss how llamafiles[1] work, and how they differ from the GGUF files. [1]: https://github.com/Mozilla-Ocho/llamafile
    comment by mc10 · 2024-03-01 · in GGUF, the Long Way Around
  12. Not as far as I know, but llamafiles are almost easier to get running than native packages. https://github.com/Mozilla-Ocho/llamafile
    comment by louismerlin · 2024-03-01 · in Ask HN: What AI assistants are already bundled for Linux?
  13. have you seen llamafile[0]? [0] https://github.com/Mozilla-Ocho/llamafile
    comment by vorticalbox · 2024-02-28 · in Show HN: I made an app to use local AI as daily driver
  14. …/ Command line (not for me): - https://ollama.com - https://privategpt.dev - https://serge.chat - https://github.com/Mozilla-Ocho/llamafile Pending to check: - https://recurse.chat Feel free to recommend more!
    comment by pentagrama · 2024-02-28 · in Show HN: I made an app to use local AI as daily driver
  15. …-p '[INST]Write a story about llamas[/INST]' [0] https://llm.datasette.io/en/stable/index.html [1] https://github.com/Mozilla-Ocho/llamafile Edit: added llm quickstart from the intro page
    comment by fbdab103 · 2024-02-27 · in Ask HN: People who switched from GPT to their own models. How was it?
  16. …use llamafile for this. It is a standalone executable that is portable across most popular OSes. https://github.com/Mozilla-Ocho/llamafile/releases So, download the executable from the releases page under assets. You want either just main…
    comment by superkuh · 2024-02-23 · in Gemma.cpp: lightweight, standalone C++ inference engine for Gemma models
  17. So... llamafile release? https://github.com/Mozilla-Ocho/llamafile
    comment by dartharva · 2024-02-23 · in Gemma.cpp: lightweight, standalone C++ inference engine for Gemma models
  18. Can we have llamafile releases as well? https://github.com/Mozilla-Ocho/llamafile
    comment by dartharva · 2024-02-21 · in Gemma: New Open Models
  19. …gpu support. on windows, it only depends on the graphics driver, thanks to our tinyBLAS library. https://github.com/Mozilla-Ocho/llamafile/releases/tag/0.6.2 By default it opens a browser tab with a chat gui. You can run it as a cli…
    comment by jart · 2024-02-17 · in Ollama is now available on Windows in preview
  20. …for locally hosting LLMs over the last few months have been amazing. I was ranting about how easy https://github.com/Mozilla-Ocho/llamafile is just a few hours ago [1]. Now I'm torn as to which one to use :) 1: Quite literally hours ago:…
    comment by ultrasaurus · 2024-02-08 · in OpenAI compatibility
  21. Slightly off topic, here is the best local llama.cpp wrapper I've run into: https://github.com/Mozilla-Ocho/llamafile You can download any .gguf model (not just the ones in their examples) and run it locally (as long as you have the ram…
    comment by depingus · 2024-02-08 · in Localllm lets you develop gen AI apps on local CPUs
  22. Have you looked at llamafile[0]? It can spin up a openAi compatible API for a while host of LLM. [0] https://github.com/Mozilla-Ocho/llamafile
    comment by vorticalbox · 2024-02-07 · in Oslo acquires ChatGPT for 110k students and teachers
  23. …collection, I recognise the output style the template-feeling elements of it, much more now. [1] https://github.com/Mozilla-Ocho/llamafile/ [2] https://news.ycombinator.com/item?id=19251755
    comment by jodrellblank · 2024-02-01 · in Show HN: Some blind hackers are bridging IRC to LMMs running locally
  24. Does anyone have any information about LLaVA-1.6 being bundled as a LLamafile? https://github.com/Mozilla-Ocho/llamafile
    comment by bsenftner · 2024-02-01 · in LLaVA-1.6: Improved reasoning, OCR, and world knowledge
  25. In my experience award for easiest to run locally will go to llamafile models https://github.com/Mozilla-Ocho/llamafile .
    comment by acd10j · 2024-01-25 · in Ollama releases Python and JavaScript Libraries
  26. …one executable LLM that works on Mac, windows and Linux. Currently pushing for application note https://github.com/Mozilla-Ocho/llamafile/pull/178 to encourage integration. Would be good to hear your thoughts on making it easier for home…
    comment by mofosyne · 2024-01-14 · in Building a fully local LLM voice assistant to control my smart home
  27. …tools that require more setup than downloading and running a single llamafile executable? [1] https://github.com/Mozilla-Ocho/llamafile/
    comment by barnabee · 2024-01-09 · in Mixtral 8x7B: A sparse Mixture of Experts language model
  28. …[0] https://hacks.mozilla.org/2023/11/introducing-llamafile/ [1] https://github.com/Mozilla-Ocho/llamafile#quickstart
    comment by noman-land · 2024-01-09 · in Mixtral 8x7B: A sparse Mixture of Experts language model
  29. You can download the llamafile executables from Mozilla's release page here: https://github.com/Mozilla-Ocho/llamafile/releases and then use the `-m` flag which lets you load any GGUF weights you want from Hugging Face. A lot of people I…
    comment by jart · 2023-12-31 · in Emacs-copilot: Large language model code completion for Emacs
  30. ….args You'd then grab the latest llamafile release binary off https://github.com/Mozilla-Ocho/llamafile/releases/ along with our zipalign program, and use it to insert the weights back into the new file: zipalign -j0 lla
    comment by jart · 2023-12-31 · in Emacs-copilot: Large language model code completion for Emacs
  31. …somehow encapsulates the model and the code used to interface with it. Is it the one listed here? https://github.com/Mozilla-Ocho/llamafile
    comment by spit2wind · 2023-12-31 · in Emacs-copilot: Large language model code completion for Emacs
  32. Mixtral-8x7B-Instruct is also available in a llamafile now: https://github.com/Mozilla-Ocho/llamafile#other-example-llam...
    comment by throwup238 · 2023-12-18 · in Many options for running Mistral models in your terminal using LLM
  33. Try llamafile https://github.com/Mozilla-Ocho/llamafile I have Mistral 7B running this way on a 10 year old laptop and it only seems to use a few GB with it's memory mapping approach.
    comment by trailbits · 2023-12-17 · in Run Mistral 7B on M1 Mac
  34. Somehow wound up installing from https://github.com/mozilla-Ocho/llamafile Don't ask me how I wound up there - I apparently followed links from this HN article.
    comment by CliffStoll · 2023-12-13 · in Bash one-liners for LLMs
  35. …a distributable DLL that only depends on KERNEL32.DLL. Oh it'll also have Mixtral support :) https://github.com/Mozilla-Ocho/llamafile/pull/82
    comment by jart · 2023-12-13 · in Bash one-liners for LLMs
  36. …to do it. I don't know how helpful the model is with screenshots though. 1. Download LLaVA from https://github.com/Mozilla-Ocho/llamafile 2. Run Whisper locally for speech to text 3. Save screenshots and send to the model, with a script…
    comment by dave1010uk · 2023-12-12 · in Show HN: Open-source macOS AI copilot using vision and voice
  37. …right now. Weights here: https://huggingface.co/jartine/llava-v1.5-7B-GGUF/tree/main Sauce here: https://github.com/mozilla-Ocho/llamafile
    comment by jart · 2023-12-11 · in Show HN: I Remade the Fake Google Gemini Demo, Except Using GPT-4 and It's Real
  38. I’ve been using llava via https://github.com/Mozilla-Ocho/llamafile which runs on any modern system.
    comment by ok_dad · 2023-12-11 · in Show HN: I Remade the Fake Google Gemini Demo, Except Using GPT-4 and It's Real
  39. Closing the loop for anyone reading this thread -- see https://github.com/Mozilla-Ocho/llamafile/issues/11 for the fix. Thanks jart!
    comment by derwiki · 2023-12-01 · in Llamafile lets you distribute and run LLMs with a single file
  40. Looks like this may be due to crowdstrike, which I also have installed on this machine: https://github.com/Mozilla-Ocho/llamafile/issues/14#issuecom...
    comment by carbocation · 2023-11-30 · in Llamafile lets you distribute and run LLMs with a single file
  41. We have an issue here tracking this: https://github.com/Mozilla-Ocho/llamafile/issues/14 Please follow that issue for updates.
    comment by jart · 2023-11-30 · in Llamafile lets you distribute and run LLMs with a single file
  42. I'm assuming you're on Apple Silicon? Please follow https://github.com/Mozilla-Ocho/llamafile/issues/11 which is tracking this. We've received multiple reports even though I personally haven't figured out how to reproduce it yet.
    comment by jart · 2023-11-30 · in Llamafile lets you distribute and run LLMs with a single file
  43. …and pass any GGUF model as an argument. > llamafile-server-0.1 -m llama-2-13b.Q8_0.gguf https://github.com/Mozilla-Ocho/llamafile/releases/tag/0.1
    comment by column · 2023-11-30 · in Llamafile lets you distribute and run LLMs with a single file
  44. …llama.cpp project upstream that you'll need in order to run these weights using llamafile. Enjoy! https://github.com/Mozilla-Ocho/llamafile/commit/865462fc465...
    comment by jart · 2023-11-30 · in Llamafile lets you distribute and run LLMs with a single file
  45. Sounds like you should download the 4.45MB llamafile-server-0.1 executable from https://github.com/Mozilla-Ocho/llamafile/releases/tag/0.1 and then run it against your existing gguf model files like this: ./llamafile-server-0.1 -m…
    comment by simonw · 2023-11-30 · in Llamafile lets you distribute and run LLMs with a single file
  46. …Please see the Gotchas section of the README for the copy/pastable commands you can run: https://github.com/mozilla-Ocho/llamafile#gotchas
    comment by jart · 2023-11-30 · in Llamafile lets you distribute and run LLMs with a single file
  47. …I'm not aware of any sh that still doesn't support APE. See the Gotchas section of the README https://github.com/mozilla-Ocho/llamafile#gotchas
    comment by jart · 2023-11-30 · in Llamafile lets you distribute and run LLMs with a single file
  48. You'd do something like this on PowerShell: curl -Lo llamafile.exe https://github.com/Mozilla-Ocho/llamafile/releases/download/0.1/llamafile-server-0.1 curl -Lo wizard.gguf https://huggingface.co/TheBloke/WizardCoder-Python-13B-V1.0-GGUF/re…
    comment by jart · 2023-11-30 · in Llamafile lets you distribute and run LLMs with a single file
  49. …tasks - the file should be everything you need. There are some gotchas to watch out for though: https://github.com/mozilla-Ocho/llamafile#gotchas
    comment by simonw · 2023-11-29 · in Llamafile lets you distribute and run LLMs with a single file
  50. …is 0xfe1c0ed4 bytes in size, which is just 30MB shy of that limit. https://github.com/Mozilla-Ocho/llamafile/commit/81c6ad3251f...
    comment by jart · 2023-11-29 · in Llamafile lets you distribute and run LLMs with a single file