ml-explore/mlx-lm
View on GitHub ↗ ★ 7,532 Python
Run LLMs with MLX
8 mentions on Hacker News by 7 people: 0 stories, 8 comments · first 2025-04-28 · latest 2026-06-17
20082026
-
Yeah, first think I looked for on the post was MLX and it wasn't there. https://github.com/ml-explore/mlx-lm Having used half the systems that Vicki mentioned, mlx was the best balance between power and ease of use. Just a pip install away.
comment by genxy · 2026-06-17 · in Running local models is good now -
It's not implemented in mlx[1] yet (or llama.cpp[2]), so it may take a while. [1] https://github.com/ml-explore/mlx-lm/pull/990 [2] https://github.com/ggml-org/llama.cpp/pull/22673
comment by dvt · 2026-05-05 · in Accelerating Gemma 4: faster inference with multi-token prediction drafters -
Get into a venv, and run: > pip3 install git+ https://github.com/ml-explore/mlx-lm.git > ./venv/bin/mlx_lm.generate --model "$MODEL" --temp 1.0 --top-p 0.95 --top-k 64 --max-tokens 128000 --prompt "Hello world" Where $MODEL is an unsloth…
comment by Confiks · 2026-04-14 · in I ran Gemma 4 as a local model in Codex CLI -
…instead. That too was broken in mlx-lm (it crashed), but has since been fixed on the main branch of https://github.com/ml-explore/mlx-lm . I unfortunately only have 16 GiB of RAM on a Macbook M1, but I just tried to run the Q8_0 GGUF…
comment by Confiks · 2026-04-13 · in I ran Gemma 4 as a local model in Codex CLI -
…https://github.com/swiss-ai/apertus-tech-report/blob/main/Ap... quantizations: available now in MLX https://github.com/ml-explore/mlx-lm (gguf coming soon, not trivial due to new architecture) model sizes: still many good dense models…
comment by lllllm · 2025-09-05 · in Apertus 70B: Truly Open - Swiss LLM by ETH, EPFL and CSCS -
…(requires the latest mlx-lm to be installed) here's the MLX-lm PR if you're interested: https://github.com/ml-explore/mlx-lm/pull/272 similarly, llama.cpp here: https://github.com/ggml-org/llama.cpp/pull/14581 Let me know if you face any…
comment by reach-vb · 2025-07-09 · in Smollm3: Smol, multilingual, long-context reasoner LLM -
…but only for Apple Silicon. mlx-lm is a llama.cpp equivalent, but built on top of mlx. https://github.com/ml-explore/mlx-lm
comment by marci · 2025-04-28 · in Tiny-LLM – a course of serving LLM on Apple Silicon for systems engineers -
…I've used to try out models: https://simonwillison.net/tags/mlx/ I mainly use MLX for LLMs (with https://github.com/ml-explore/mlx-lm and my own https://github.com/simonw/llm-mlx which wraps that), vision LLMs (via…
comment by simonw · 2025-04-28 · in Tiny-LLM – a course of serving LLM on Apple Silicon for systems engineers