Hacker GitHub

← All repos

mozilla-ai/llamafile

View on GitHub ↗ ★ 26,545 C++

Distribute and run LLMs with a single file.

112 mentions on Hacker News by 82 people: 10 stories, 102 comments · first 2023-11-29 · latest 2026-10-27

Mentions per year 112 mentions · busiest year 2024 (62)
20082026
AllStoriesComments NewestTopOldest
  1. I like to use llamafile for cases like this: https://github.com/mozilla-ai/llamafile
    comment by majorchord · 2026-10-27 · in Survival LLM: AI assistant and emergency library that runs on your own computer
  2. sounds loke a job for llamafile! https://github.com/mozilla-ai/llamafile
    comment by sellmesoap · 2026-05-15 · in Show HN: Needle: We Distilled Gemini Tool Calling into a 26M Model
  3. …also recommend llama-file, it’s a one file executable for any OS that includes your chosen model: https://github.com/mozilla-ai/llamafile?tab=readme-ov-file It’s truly open source, backed by Mozilla, openly uses llama.cpp and was created…
    comment by brabel · 2026-04-16 · in The local LLM ecosystem doesn’t need Ollama
  4. …framework-based executable. We're hoping to build some bridges between the two projects as well. https://github.com/mozilla-ai/llamafile
    comment by JohnPDickerson · 2026-04-06 · in Show HN: Ghost Pepper – Local hold-to-talk speech-to-text for macOS
  5. …and spin up an OpenAI compatible API endpoint. It even downloads the models for you. [0]: https://github.com/mozilla-ai/llamafile
    comment by randusername · 2026-03-13 · in Can I run AI locally?
  6. Mozilla is working on it again, and they're asking for input: https://github.com/mozilla-ai/llamafile/discussions/809
    comment by stlhood · 2025-12-14 · in llamafile: Distribute and Run LLMs with a Single File
  7. …Wine. If binfmt_misc is causing problems, the proper solution is to register the executable type. https://github.com/mozilla-ai/llamafile#linux describes steps. I made myself a package containing /usr/bin/ape and the following…
    comment by chrismorgan · 2025-11-01 · in Llamafile Returns
  8. …is going to bring it back. This announcement didn't even come with a new code commit, just a wish. https://github.com/mozilla-ai/llamafile/commits/main/
    comment by benatkin · 2025-11-01 · in Llamafile Returns
  9. Also see https://github.com/Mozilla-Ocho/llamafile .
    comment by TYPE_FASTER · 2025-09-08 · in Experimenting with Local LLMs on macOS
  10. …as well on Windows due to the executable file size limit but seems great for Mac/Linux flavors. https://github.com/Mozilla-Ocho/llamafile
    comment by vonneumannstan · 2025-09-08 · in Experimenting with Local LLMs on macOS
  11. …Every 5 Seconds) https://geppetto.app I contributed "whisperfile" as a result of this work: * https://github.com/Mozilla-Ocho/llamafile/tree/main/whisper.... * https://github.com/cjpais/whisperfile if you ever want to chat about making…
    comment by sipjca · 2025-07-30 · in Fast
  12. https://github.com/Mozilla-Ocho/llamafile Cost: $0 Currently, I treat all generated code as "sample code" which is not of much use and a waste of time but let's see what the future brings. I use llama to generate "boilerplate" in simple…
    comment by mmphosis · 2025-07-17 · in Ask HN: What's Your Useful Local LLM Stack?
  13. Seconded for Llamafile, here is a link for references https://github.com/Mozilla-Ocho/llamafile . It indeed is working on all major platforms and its tooling allows easy creating of new llamafiles with new models. The only caveat is…
    comment by romperstomper · 2025-07-02 · in Show HN: Local LLM Notepad – run a GPT-style model from a USB stick
  14. https://github.com/Mozilla-Ocho/llamafile not working ;( gemma-3n-E4B-it-Q8_0 import_cuda_impl: initializing gpu module... get_rocm_bin_path: note: hipcc not found on $PATH [...] llama_model_load: error loading model: error loading model…
    comment by eabeezxjc · 2025-06-27 · in Introducing Gemma 3n
  15. I thought llamafile was supposed to be the solution to "too hard to work with"? https://github.com/Mozilla-Ocho/llamafile
    comment by ranger_danger · 2025-04-21 · in Local LLM inference – impressive but too hard to work with
  16. …Actually Portable Executable (APE) binaries, please see this reference Mozilla put together: https://github.com/Mozilla-Ocho/llamafile/?tab=readme-ov-fil...
    comment by jart · 2025-04-14 · in Show HN: Resurrecting Infocom's Unix Z-Machine with Cosmopolitan
  17. …ps://blob.localscore.ai/localscore-0.9.2 380MB? I remember llamafile being only a few megabytes. From https://github.com/Mozilla-Ocho/llamafile/releases , looks like it steadily grew from adding support for GPUs on more platforms, up to…
    comment by chrismorgan · 2025-04-07 · in Show HN: LocalScore – Local LLM Benchmark
  18. …in their efforts to ensure this special moment in history gets archived for future generations! https://github.com/Mozilla-Ocho/llamafile/
    comment by jart · 2025-03-16 · in Big LLMs weights are a piece of history
  19. Try one and find out. Look at https://github.com/Mozilla-Ocho/llamafile/ Quickstart section; download a single cross-platform ~3.7GB file and execute it, it starts a local model, local webserver, and you can query it. See it demonstrated…
    comment by jodrellblank · 2025-02-01 · in How to Run DeepSeek R1 671B Locally on a $2000 EPYC Server
  20. Llamafile: https://github.com/Mozilla-Ocho/llamafile
    comment by bestcoder69 · 2025-01-31 · in Llama.cpp supports Vulkan. why doesn't Ollama?
  21. Firefox released a very easy to deploy local LLM around a year ago. https://github.com/Mozilla-Ocho/llamafile https://ai-guide.future.mozilla.org/content/running-llms-loc... It works.
    comment by getwiththeprog · 2025-01-01 · in Orbit by Mozilla
  22. …persistent badge on every page) and then make it ~truly privacy respecting by moving locally. [0] https://github.com/Mozilla-Ocho/llamafile [1] https://github.com/mozilla/translations
    comment by usernomdeguerre · 2024-12-31 · in Orbit by Mozilla
  23. You might also try https://github.com/Mozilla-Ocho/llamafile , which may have better CPU-only performance than ollama. It does require you to grab .gguf files yourself (unless you use one of their prebuilts in which case it comes with the…
    comment by yjftsjthsd-h · 2024-11-29 · in Llama.cpp guide – Running LLMs locally on any hardware, from scratch
  24. …know if this would help your case or not, but jart’s llamafile seems like it would be useful [6] https://github.com/Mozilla-Ocho/llamafile
    comment by phatskat · 2024-11-22 · in The tragedy of running an old Node project
  25. https://github.com/Mozilla-Ocho/llamafile Wait.. there is one binary that executes just fine on half a dozen platforms? What wizardry is this? edit: Their default LLM worked great on Windows. Fast inference on my 2080ti. You have to pass…
    comment by qingcharles · 2024-11-10 · in Everything I've learned so far about running local LLMs
  26. I wrote https://github.com/Mozilla-Ocho/llamafile/blob/main/llamafil... and it does a reasonable job highlighting without breaking for all the .tex files I could find on my hard drive. My goal is to hopefully cover 99.9% of real world…
    comment by jart · 2024-11-02 · in Weird Lexical Syntax
  27. Llamafile literally one file https://github.com/Mozilla-Ocho/llamafile
    comment by FergusArgyll · 2024-10-15 · in Ask HN: Why is AI/LLMs so hard to install? Where's the one click installers?
  28. > then I recommend https://github.com/Mozilla-Ocho/llamafile which ships as a single file with no dependencies and runs on CPU with great performance. Like, such great performance that I've mostly given up on GPU for LLMs. It was a game…
    comment by seu · 2024-10-12 · in Run Llama locally with only PyTorch on CPU
  29. …goal is > Run and explore Llama models locally with minimal dependencies on CPU then I recommend https://github.com/Mozilla-Ocho/llamafile which ships as a single file with no dependencies and runs on CPU with great performance. Like, such…
    comment by yjftsjthsd-h · 2024-10-11 · in Run Llama locally with only PyTorch on CPU
  30. Mozilla has a project called Llamafile ( https://github.com/Mozilla-Ocho/llamafile ) that's based on Cosmopolitan libc. And they do regularly publish popular models repackaged in that format on Hugging Face:…
    comment by int_19h · 2024-10-02 · in The Fastest Mutexes
  31. …I was running llava-v1.5-7b-q4.llamafile which is the example that they recommend trying first at https://github.com/Mozilla-Ocho/llamafile Groq looks interesting and might be a better option for me. Thank you.
    comment by senkora · 2024-09-23 · in Forget ChatGPT: why researchers now run small AIs on their laptops
  32. …quantization. You can use other models with llamafile as well. They have some prebuilt ones: https://github.com/Mozilla-Ocho/llamafile?tab=readme-ov-file... You can also search for other llamafiles for other models on HuggingFace by using…
    comment by noman-land · 2024-09-22 · in Forget ChatGPT: why researchers now run small AIs on their laptops
  33. …https://ai-guide.future.mozilla.org/content/running-llms-loc... https://github.com/mozilla-Ocho/llamafile https://justine.lol/matmul/ and parallelize them with https://github.com/b4rtaz/distributed-llama Obviously this nee
    comment by LargoLasskhyfv · 2024-09-21 · in Ask HN: How can I experiment with LLMs with a old machine?
  34. …and try it out in just moments. https://future.mozilla.org/builders/news_insights/introducin... https://github.com/Mozilla-Ocho/llamafile They even have whisperfiles now, which is the same thing but for whisper.cpp, aka real-time voice…
    comment by noman-land · 2024-09-21 · in Forget ChatGPT: why researchers now run small AIs on their laptops
  35. …from Simon Willison: https://simonwillison.net/2024/Aug/19/whisperfile/ Command line options: https://github.com/Mozilla-Ocho/llamafile/issues/544#issueco...
    comment by tbran · 2024-09-16 · in Speech Dictation Mode for Emacs
  36. With Mozilla's llamafile you can run LLMs locally without installing anything: https://github.com/Mozilla-Ocho/llamafile
    comment by PhilippGille · 2024-09-05 · in Yi-Coder: A Small but Mighty LLM for Code
  37. …(And thank you to Mozilla for financially supporting her work). See https://justine.lol/matmul/ and https://github.com/mozilla-Ocho/llamafile
    comment by simpaticoder · 2024-07-18 · in Mistral NeMo
  38. If you mean LLM in general, maybe try llamafile first https://github.com/Mozilla-Ocho/llamafile
    comment by sva_ · 2024-07-16 · in Codestral Mamba
  39. For pure self-hosting, I'd look into Ollama (ollama.com) or llamafile ( https://github.com/Mozilla-Ocho/llamafile ) on the LLM side and then picking a UI such as https://openwebui.com/ or one of the many in this list:…
    comment by eschnou · 2024-07-05 · in Ask HN: Best way to deploy an LLM for 100s of user
  40. Llamafile: https://github.com/Mozilla-Ocho/llamafile
    comment by PhilippGille · 2024-06-24 · in Cosmopolitan v3.5
  41. Depending on your internet speed, with llamafile you can do it even faster. Go to https://github.com/Mozilla-Ocho/llamafile , find Quickstart section and have fun. Scroll down a bit further for more models.
    comment by butz · 2024-06-23 · in How to run an LLM on your PC, not in the cloud, in less than 10 minutes
  42. The absolute easiest way is https://github.com/Mozilla-Ocho/llamafile Just download a single file and run it.
    comment by SahAssar · 2024-06-15 · in Cost of self hosting Llama-3 8B-Instruct
  43. …have to be integrated in the browser, though? Because Mozilla already distributes offline LLMs: https://github.com/Mozilla-Ocho/llamafile .
    comment by tredre3 · 2024-06-11 · in Firefox 127
  44. OpenBSD and Llamafile go hand in hand. https://github.com/Mozilla-Ocho/llamafile
    comment by WhackyIdeas · 2024-05-24 · in Self-hosting keeps your private data out of AI models
  45. See llamafile ( https://github.com/Mozilla-Ocho/llamafile ), a standalone packaging of llama.cpp that runs an LLM locally. It will use the GPU, but falls back on the CPU. CPU-only performance of small, quantized models is still pretty…
    comment by ynniv · 2024-05-22 · in Ask HN: Which LLMs can run locally on most consumer computers
  46. I developed this originally for llamafile, which was included in the last two releases: https://github.com/Mozilla-Ocho/llamafile/releases/tag/0.8.2 Now we're upstreaming it to the llama.cpp project. There are other performance…
    comment by jart · 2024-05-15 · in New exponent functions that make SiLU and SoftMax 2x faster, at full accuracy
  47. I think llamafile supports openai compatible api.. https://github.com/Mozilla-Ocho/llamafile
    comment by indigodaddy · 2024-05-01 · in Show HN: I'm 16 and building an AI based startup called Factful with friends
  48. …the ecosystem, but I think you should be able to run this new model as is against a llamafile? [0] https://github.com/Mozilla-Ocho/llamafile
    comment by fbdab103 · 2024-04-10 · in Mistral AI Launches New 8x22B MOE Model
  49. …(HA). Maybe someone is working on this already? TTS: https://github.com/SYSTRAN/faster-whisper LLM: https://github.com/Mozilla-Ocho/llamafile/releases LLM: https://huggingface.co/TheBloke/Nous-Hermes-2-Mixtral-8x7B-D... HA:…
    comment by thangalin · 2024-04-04 · in Apple explores home robotics as potential 'next big thing'
  50. While I did not succeed in making the matmul code from https://github.com/Mozilla-Ocho/llamafile/blob/main/llamafil... work in isolation, I compared eigen, openblas, and mkl: https://gist.github.com/Dobiasd/e664c681c4a7933ef5d2df7caa87...…
    comment by Dobiasd · 2024-04-04 · in LLaMA now goes faster on CPUs