Hacker GitHub

← All repos

jundot/omlx

View on GitHub ↗ ★ 23,287 Python

LLM inference server with continuous batching & SSD caching for Apple Silicon — managed from the macOS menu bar

15 mentions on Hacker News by 9 people: 7 stories, 8 comments · first 2026-02-13 · latest 2026-06-29

Mentions per year 15 mentions · busiest year 2026 (15)
20082026
AllStoriesComments NewestTopOldest
  1. …can have a useful app. Anyway, if you're getting started on a Mac, I'd suggest trying out oMLX ( https://github.com/jundot/omlx ) before messing with llama.cpp. In particular they have community benchmarks so you can see what kind of…
    comment by blopker · 2026-06-29 · in Qwen 3.6 27B is the sweet spot for local development
  2. oMLX ( https://github.com/jundot/omlx ) makes running the mlx inference server quite easy for those interested in UI-based hosting. oMLX also supports mtp or dflash drafting.
    comment by namnnumbr · 2026-06-12 · in How to setup a local coding agent on macOS
  3. ▲ 2
    OMLX v0.3.9 Stable Merges Native MTP (Multi-Token Prediction)
    story by febed · 2026-05-22 · 0 comments
  4. ▲ 4
    OMLX: Local LLM Server for Apple Silicon Macs
    story by steveharing1 · 2026-05-11 · 0 comments
  5. https://github.com/jundot/omlx note: 27b is going to be slow; use the 35b MoE if you want decent token/sec speed.
    comment by jonaustin · 2026-04-27 · in Running local LLMs offline on a ten-hour flight
  6. Try https://github.com/jundot/omlx I've been using qwen3.5 (122b) with claude code for months, and it's definitely dumber than sonnet/opus, but it works through things reasonably well (i.e. writes half-decent code and tool calls usually…
    comment by jonaustin · 2026-04-27 · in Running local LLMs offline on a ten-hour flight
  7. …latest qwen model you can fit in unified ram. Hooking up Claude Code to it is trivial with omlx. https://github.com/jundot/omlx
    comment by jonaustin · 2026-04-24 · in I cancelled Claude: Token issues, declining quality, and poor support
  8. how about the others: - vLLM https://vllm.ai/ ? - oMLX https://github.com/jundot/omlx ?
    comment by dragochat · 2026-04-16 · in The local LLM ecosystem doesn’t need Ollama
  9. I run mlx models with omlx[1] on my mac and it works really well. [1] https://github.com/jundot/omlx
    comment by c2k · 2026-04-05 · in Gemma 4 on iPhone
  10. …Studio's model directory, so you don't need to re-download anything. 100% open source (Apache 2.0): https://github.com/jundot/omlx
    comment by jundot · 2026-03-04 · in Show HN: oMLX – SSD-backed KV cache cuts coding agent TTFT from 90s to 1s on Mac
  11. ▲ 4
    Show HN: oMLX – SSD-backed KV cache cuts coding agent TTFT from 90s to 1s on Mac
    story by jundot · 2026-03-04 · 1 comment
  12. ▲ 3
    Show HN: oMLX – coding agents on local LLMs without the painful reprefill
    story by jundot · 2026-02-24 · 0 comments
  13. ▲ 1
    Show HN: oMLX – Native Mac inference server that persists KV cache to SSD
    story by jundot · 2026-02-19 · 0 comments
  14. ▲ 2
    OMLX – Ollama for MLX (LLM Inference Server for Apple Silicon)
    story by fintechie · 2026-02-15 · 0 comments
  15. ▲ 3
    OMLX – LLM Inference Server for Apple Silicon (Ollama for MLX)
    story by fintechie · 2026-02-13 · 0 comments