jundot/omlx
View on GitHub ↗ ★ 23,287 Python
LLM inference server with continuous batching & SSD caching for Apple Silicon — managed from the macOS menu bar
15 mentions on Hacker News by 9 people: 7 stories, 8 comments · first 2026-02-13 · latest 2026-06-29
20082026
-
…can have a useful app. Anyway, if you're getting started on a Mac, I'd suggest trying out oMLX ( https://github.com/jundot/omlx ) before messing with llama.cpp. In particular they have community benchmarks so you can see what kind of…
comment by blopker · 2026-06-29 · in Qwen 3.6 27B is the sweet spot for local development -
oMLX ( https://github.com/jundot/omlx ) makes running the mlx inference server quite easy for those interested in UI-based hosting. oMLX also supports mtp or dflash drafting.
comment by namnnumbr · 2026-06-12 · in How to setup a local coding agent on macOS -
https://github.com/jundot/omlx note: 27b is going to be slow; use the 35b MoE if you want decent token/sec speed.
comment by jonaustin · 2026-04-27 · in Running local LLMs offline on a ten-hour flight -
Try https://github.com/jundot/omlx I've been using qwen3.5 (122b) with claude code for months, and it's definitely dumber than sonnet/opus, but it works through things reasonably well (i.e. writes half-decent code and tool calls usually…
comment by jonaustin · 2026-04-27 · in Running local LLMs offline on a ten-hour flight -
…latest qwen model you can fit in unified ram. Hooking up Claude Code to it is trivial with omlx. https://github.com/jundot/omlx
comment by jonaustin · 2026-04-24 · in I cancelled Claude: Token issues, declining quality, and poor support -
how about the others: - vLLM https://vllm.ai/ ? - oMLX https://github.com/jundot/omlx ?
comment by dragochat · 2026-04-16 · in The local LLM ecosystem doesn’t need Ollama -
I run mlx models with omlx[1] on my mac and it works really well. [1] https://github.com/jundot/omlx
comment by c2k · 2026-04-05 · in Gemma 4 on iPhone -
…Studio's model directory, so you don't need to re-download anything. 100% open source (Apache 2.0): https://github.com/jundot/omlx
comment by jundot · 2026-03-04 · in Show HN: oMLX – SSD-backed KV cache cuts coding agent TTFT from 90s to 1s on Mac