Hacker GitHub

← All repos

mlc-ai/web-llm

View on GitHub ↗ ★ 19,348 TypeScript

High-performance In-browser LLM Inference Engine

22 mentions on Hacker News by 21 people: 4 stories, 18 comments · first 2023-04-15 · latest 2026-09-02

Mentions per year 22 mentions · busiest year 2025 (9)
20082026
AllStoriesComments NewestTopOldest
  1. ▲ 145
    WebLLM: high-performance in-browser LLM inference engine
    story by saikatsg · 2026-09-02 · 25 comments
  2. ▲ 1
    WebLLM is a high-performance in-browser LLM inference engine
    story by doener · 2026-05-02 · 0 comments
  3. …from this that it will be delayed unfortunately. However, WebLLM (a library, not actual Web API) https://github.com/mlc-ai/web-llm is more capable and will already work using WebGPU.
    comment by ilaksh · 2026-04-30 · in Mozilla's opposition to Chrome's Prompt API
  4. …here: https://simonwillison.net/tags/transformers-js/ You can also get this working with web-llm - https://github.com/mlc-ai/web-llm - here's my write-up of a demo that uses that: https://simonwillison.net/2024/Nov/29/structured-generation-…
    comment by simonw · 2025-09-08 · in Experimenting with Local LLMs on macOS
  5. …with anythingllm for the UI but I can see the appeal of having it all just run in the browser. https://github.com/mlc-ai/web-llm https://github.com/ngxson/wllama
    comment by craftkiller · 2025-09-08 · in Experimenting with Local LLMs on macOS
  6. I think what you want is this: https://github.com/mlc-ai/web-llm
    comment by tmdetect · 2025-09-08 · in Experimenting with Local LLMs on macOS
  7. Yes. MLC's inference engine runs on WebGPU/WASM. https://github.com/mlc-ai/web-llm-chat https://github.com/mlc-ai/mlc-llm https://github.com/mlc-ai/web-llm
    comment by SparkyMcUnicorn · 2025-09-08 · in Experimenting with Local LLMs on macOS
  8. Looks like this is a wrapper around: https://github.com/mlc-ai/web-llm Which has a full web demo: https://chat.webllm.ai/
    comment by maxmcd · 2025-08-02 · in Show HN: WebGPU enables local LLM in the browser – demo site with AI chat
  9. …LLMs without installing anything. Works across most modern browsers!!! Credits: [Web LLM Github]( https://github.com/mlc-ai/web-llm ) Note: The notebook takes 3-4 four minutes to load the LLM for the first time. Next time onwards it will…
    comment by gopisuvanam · 2025-02-19
  10. How does it compare to WebLLM ( https://github.com/mlc-ai/web-llm )?
    comment by hazelnut · 2025-01-22 · in Show HN: BrowserAI – Run LLMs directly in browser using WebGPU (open source)
  11. …[0] and transformers.js [1]. What exactly are you offering on top of those two libraries? [0]: https://github.com/mlc-ai/web-llm [1]: https://huggingface.co/docs/transformers.js/en/index
    comment by Philpax · 2025-01-22 · in Show HN: BrowserAI – Run LLMs directly in browser using WebGPU (open source)
  12. …blocks to accomplish this right from the browser, especially since the models are so small. https://github.com/mlc-ai/web-llm https://huggingface.co/docs/transformers.js/en/index You do have to worry about WebGPU compatibility in browsers…
    comment by MarioMan · 2025-01-22 · in Ask HN: Is anyone doing anything cool with tiny language models?
  13. …https://huggingface.co/docs/transformers.js/en/index You can also take a look at Webllm library. https://github.com/mlc-ai/web-llm In the future, google will probably load Gemini nano into chrome, right now it's in beta -…
    comment by orenha · 2024-10-27 · in Show HN: I built a paragraph rewriter that runs 100% in the browser
  14. …Mobile, and [Wllama]( https://github.com/ngxson/wllama ) together with [Web-LLM]( https://github.com/mlc-ai/web-llm ), along with LLMs of <1B parameters, allowed it! If you're curious, here's the HN post I made about it a year ago: https://
    comment by felladrin · 2024-10-14 · in Show HN: Self-hosteable browser-based minimalist web-searching platform
  15. …to download to the local user's browser (combined with my bandwidth costs) aren't exorbitant. [1] https://github.com/mlc-ai/web-llm
    comment by HanClinto · 2024-07-10 · in MobileLLM: Optimizing Sub-Billion Parameter Language Models for On-Device Use
  16. check out https://github.com/mlc-ai/web-llm IMO the main limitation is access to powerful GPUs for running models locally and the size of some models causing UX problems with cold starts
    comment by Crier1002 · 2024-07-08 · in Ask HN: Browser-Based LLM Models?
  17. https://github.com/mlc-ai/web-llm basically bringing the power of llms to web browsers via WebGPU. i think it’s useful because it’s better for privacy as nothing leaves your browser. having that said i dont see how this is feasible yet on…
    comment by Crier1002 · 2024-05-13 · in Ask HN: What is the project that you are most excited about today?
  18. It's a wrapper of https://github.com/mlc-ai/web-llm
    comment by low_tech_punk · 2024-05-04 · in Show HN: I built a free in-browser Llama 3 chatbot powered by WebGPU
  19. …off-grid communities easy access to lightweight LLMs... one idea is to use something like: https://github.com/mlc-ai/web-llm to deliver it in a browser
    comment by n8fr8too · 2024-03-05 · in LibraryBox
  20. Maybe they're talking about https://github.com/mlc-ai/mlc-llm which is used for web-llm ( https://github.com/mlc-ai/web-llm )? Seems to be using TVM.
    comment by capableweb · 2023-09-26 · in Ollama for Linux – Run LLMs on Linux with GPU Acceleration
  21. ▲ 276
    Web LLM – WebGPU Powered Inference of Large Language Models
    story by summarity · 2023-04-15 · 80 comments
  22. ▲ 6
    Web LLM
    story by crowwork · 2023-04-15 · 0 comments