antirez/ds4 on Hacker News

…<- Qwen3.8-Flash-Next. I use this (tuned) and I cannot understate how satisfied I am. https://github.com/antirez/ds4 <- DeepSeek 4 family + more.

comment by MadameMinty · 2026-10-13 · in Janas: Run LLMs larger than RAM by streaming experts from SSD

Projects like DwarfStar https://github.com/antirez/ds4 really lower the hardware bar a lot so Deepseek 4.1 flash and other mixture of expert models can run on consumer hardware. There are also other inference providers who make their money…

comment by mrinterweb · 2026-10-08 · in Why isn't the industry freaking out about DeepSeek 4.1 Flash?

…supports this, curious how it compares, but I use the q4 quant daily and it works really well. https://github.com/antirez/ds4/blob/main/docs/MODELS.md#qwen...

comment by kamranjon · 2026-10-04 · in Run Qwen 3.8 Flash Next (125B) on consumer hardware (RTX 4090) at 100T/s

from the project https://github.com/antirez/ds4 it was said that: > It must be more a working template for the biggest use cases, without trying to cover every possible setup. this is such a great quote about building software in the AI…

comment by westoque · 2026-10-03 · in From the creator of Redis; run LLM locally with ds4

https://github.com/antirez/ds4 The project GitHub page is a much better introduction for the hn crowd.

comment by twoodfin · 2026-10-02 · in From the creator of Redis; run LLM locally with ds4

…TQ to allow for 1m context lengths on a 128 gb MacBook M5 Max when using Qwen 3.8 flash next ( https://github.com/antirez/ds4/pull/1115 if you are interested). If I get bored I might port over the Metal kernels from oMLX -- the speed…

comment by vlowther · 2026-10-02 · in From the creator of Redis; run LLM locally with ds4

…floats per layer), then run against the stock weights. Antirez's DS4 already supports this: https://github.com/antirez/ds4/blob/8db1d1d155cb0400a86a86b9... Abliterated weights are just a bad habit we've gotten into. It's also deeply…

comment by wren6991 · 2026-09-20 · in Pirate Face Rescues LLM Models from Deletion

Hey there! I do the same but I use dwarfstar at a 2-bit quant: https://github.com/antirez/ds4 I'm curious if you've tried dwarfstar and decided to move to llama.cpp and 3 bit quants or what made you go that route instead? I've been using…

comment by kamranjon · 2026-09-09 · in DeepSeek launching v4.1 flash cheaper and more capable than v4 pro

…sinsider.com/ai-writing-all-startup-code-... [1] https://x.com/bcherny/status/2004897269674639461 [2] https://github.com/antirez/ds4

comment by nl · 2026-09-02 · in How accurate have Ed Zitron's AI skeptic predictions been?

Primarily used ds4[0] by antirez [0] https://github.com/antirez/ds4

comment by jumploops · 2026-09-02 · in My local model setup on an M4 Pro Mac Mini

…in between 96GB and 256GB, though. For instance, I believe Deepseek v4 flash runs a lot faster on ( https://github.com/antirez/ds4 ) with 128GB than 96. I assume the higher SKUs have low enough sales that Apple doesn't want to be…

comment by kristianp · 2026-09-01 · in Apple caught off guard by AI demand for Mac Mini and Mac Studio

…are not considering high-optimized inference engine, SSD-streaming approach and other optimization https://github.com/antirez/ds4

comment by cecinuga · 2026-08-28

…exactly straight out of one of the labs -- it's heavily quantized -- but have a look at [1]. [1] https://github.com/antirez/ds4

comment by fghorow · 2026-08-26 · in Apple introduces M6 and M5 Ultra

…multi-user serving. I'm not sure whether that qualifies as local, but it's at least not a rack. [1] https://github.com/antirez/ds4 [2] https://github.com/JustVugg/colibri

comment by wren6991 · 2026-08-14 · in GLM-5.3: Frontier coding with emergent cyber capabilities

DwarfStar ( https://github.com/antirez/ds4 ) supports GLM 5.2 and DeepSeek. Not only for toying, but for getting work done.

comment by bertili · 2026-08-14 · in GLM-5.3: Frontier coding with emergent cyber capabilities

This also works great: https://github.com/antirez/ds4

comment by complex1314 · 2026-08-08 · in DeepSeek V4 Flash 0731

https://github.com/antirez/ds4/pull/484 I assume this is parallel work.

comment by wmf · 2026-08-04 · in DeepSeek V4 Flash on a Single AMD MI300X

Looks like a couple of people are already on it: https://github.com/antirez/ds4/issues/635

comment by jtbaker · 2026-07-31 · in DeepSeek-V4-Flash Update

Parent is referring to https://github.com/antirez/ds4

comment by genxy · 2026-07-31 · in DeepSeek-V4-Flash Update

Weights are yet to be released (maybe in 24H, Deepseek has track record of releasing same day). https://github.com/antirez/ds4 is often mentioned for DS4F on Mac, but 64GB is likely not enough to achieve reasonable speeds (official weights…

comment by petu · 2026-07-31 · in DeepSeek-V4-Flash Update

https://github.com/antirez/ds4 coming out at the same time I started a new job and they gave me an m5 max a few months ago was the lightbulb moment for me.

comment by jtbaker · 2026-07-30 · in Show HN: Open-source engine running Gemma 4 26B in 2 GB RAM on any M-series Mac

…star and flash-moe. they do similar things with bigger models https://github.com/JustVugg/colibri https://github.com/antirez/ds4 https://github.com/danveloper/flash-moe

comment by gitpusher42 · 2026-07-29 · in Show HN: Open-source engine running Gemma 4 26B in 2 GB RAM on any M-series Mac

…memory, you should be able to run a DeepSeek V4 Flash quantisation at 7–10 t/s, for example with: https://github.com/antirez/ds4 or https://github.com/steadfastgaze/MoEspresso (my engine). The routed experts needed for the next tokens that…

comment by hacklas · 2026-07-29 · in Show HN: Open-source engine running Gemma 4 26B in 2 GB RAM on any M-series Mac

I'm a bit curious why not running DeepSeek V4 on top of https://github.com/antirez/ds4 . I think the results could be really good.

comment by c4pt0r · 2026-07-20 · in Nativ: Run frontier open models locally on your Mac

I wonder how it’s compared to https://github.com/antirez/ds4

comment by para_parolu · 2026-07-20 · in 6.4x faster than llama.cpp, 3.9x faster than MLX

DeepSeek V4 Flash with DwarfStar: https://github.com/antirez/ds4 The 2 bit quants are really good. I have a lot of memory so I can squeeze it all in at ~80gb.

comment by kamranjon · 2026-07-14 · in Bonsai 27B: A 27B-Class model that runs on a phone

…seems pretty effective. I've been really enjoying it. [1]: https://arxiv.org/abs/2607.05147 [2]: https://github.com/antirez/ds4/pull/502 [3]: https://arxiv.org/html/2406.00045v2 [4]: https://github.com/antirez/ds4/pull/148 [5]:…

comment by kamranjon · 2026-07-13 · in Control the Ideas, Not the Code

ds4 is pretty optimized for this btw - https://github.com/antirez/ds4

comment by ukd1 · 2026-07-11 · in Show HN: Getting GLM 5.2 running on my slow computer

…can absolutely run DS4 Flash on a 128gb machine - I have it running on my Strix Halo box right now. https://github.com/antirez/ds4

comment by Catloafdev · 2026-07-10 · in Hands-On with the AMD Ryzen AI Halo

Counterpoint, I'm just a happy user of this local LLM [1] from antirez (of Redis fame). [1] https://github.com/antirez/ds4

comment by fghorow · 2026-07-10 · in Apple Silicon Exec Explains Mac Mini AI Demand and On-Device Future

Antirez has a GLM 5.2 branch now in dwarfstar: https://github.com/antirez/ds4/tree/glm5.2 It heavily utilizes ssd streaming from my understanding and I think he mentioned getting some semi usable speeds on a 128gb m5 mbp.

comment by kamranjon · 2026-07-10 · in Show HN: Getting GLM 5.2 running on my slow computer

Try using DwarfStar 4 and use the --power flag: https://github.com/antirez/ds4#reducing-heat-power-usage-and...

comment by somewhatrandom9 · 2026-06-29 · in Qwen 3.6 27B is the sweet spot for local development

…use of the ram and use sparse MoE. For example, DeepSeek-V4-Flash will fit, served by DwarfStar ( https://github.com/antirez/ds4 ). One will probably improve 2x the token/sec speed, given DS4F 13B activated params in the MoE are ~1/2 of…

comment by ljosifov · 2026-06-29 · in Qwen 3.6 27B is the sweet spot for local development

…excited about Antirez's (Redis author) work on DS4, especially with 4bit quant using two machines: https://github.com/antirez/ds4 Right now the speed isn't good for GLM 5.2, Deepseek V4 Flash speed is okay for me (actually reading the…

comment by pixelpoet · 2026-06-28 · in AMD Strix Halo RDMA Cluster Setup Guide

…use, will mostly sit idle. 1. https://srinathh.medium.com/mid-size-local-models-are-now-co... 2. https://github.com/antirez/ds4

comment by sfifs · 2026-06-27 · in Ask HN: MacBook vs. Dedicated GPU for LLM

I think most people running deepseek locally are using DS4: https://github.com/antirez/ds4 Which provides a 2-bit quant and a mixed 2-bit/4-bit quant - which range from 80gb to 97gb Deepseek is particularly well suited for quantization and…

comment by kamranjon · 2026-06-26 · in Apple raises prices of MacBooks, iPads

Related: https://github.com/antirez/ds4 https://news.ycombinator.com/item?id=48142108

comment by peter_d_sherman · 2026-06-22 · in 284B Model Shouldn't Fit on Your Laptop. It Does [video]

Inspired by [ https://github.com/antirez/ds4 ](ds4), [ https://github.com/CerebrasResearch/reap ](REAP) and [ https://huggingface.co/0xSero/DeepSeek-V4-Flash-162B ](OxSero's Deepseek v4 reap) I wanted to push and see how much performance…

comment by sleepyeldrazi · 2026-06-22 · in DeepSeek V4 Flash optimized framework and model variants for DGX Spark

…can stand in the for the giants on all but the most challenging and/or hands-off coding tasks. [1]: https://github.com/antirez/ds4

comment by wolttam · 2026-06-21 · in Two Qwen3 models on one DGX Spark: the residency math

indeed, there's even a (pretty solid) custom server just for DS4 https://github.com/antirez/ds4 -- works very well on high-RAM Macs

comment by KerrAvon · 2026-06-20 · in LLMs Are Complicated Now

https://github.com/antirez/ds4

comment by shsh1312 · 2026-06-19 · in Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

…new Mac Studio and MacBook Pro allow optimized local inference through open-source tools such as: https://github.com/antirez/ds4

comment by shsh1312 · 2026-06-19 · in Ask HN: What tools are you using for AI-assisted code review?

ds4: prefill: 150.18 t/s, generation: 48.47 t/s together with https://github.com/antirez/ds4 + RTX Pro 6000, local provider :) I guess you could rent one too, get similar speed. https://gist.github.com/embedding-shapes/eceefa1b2699dca448f..…

comment by embedding-shape · 2026-06-16 · in Anthropic's Safety Superpower

…DeepSeek 4 Flash these are not mutually exclusive anymore. DS4 has set the bar for me these days. https://github.com/antirez/ds4

comment by jtbaker · 2026-06-16 · in Running local models is good now

…have some benefit from better use of the resident experts cache and/or of SSD transfer bandwidth.) https://github.com/antirez/ds4/issues/275 seems to provide intriguing rough results while https://github.com/antirez/ds4/issues/314 is a…

comment by zozbot234 · 2026-06-16 · in Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

Since you have 512GB, might be worth looking into running deepseek4: https://github.com/antirez/ds4

comment by russelg · 2026-06-16 · in Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

If you have a 128GB Mac you really ought to try out: https://github.com/antirez/ds4 by the creator of redis. This is probably as close to it gets to state-of-the-art local LLM + agentic coding.

comment by brycesub · 2026-06-15 · in Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

…between processes" or some other BS. Performance as described in the tables in the readme here: https://github.com/antirez/ds4 ...with a bit less than half that at "low power" (30w). Both are usable.

comment by Lwerewolf · 2026-06-15 · in Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

Waiting for this https://github.com/antirez/ds4 to stabilize for strix halo.

comment by christkv · 2026-06-15 · in Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

…harnesses. It's a bit slow compared to the other models but so good that I'm willing to put up :-) https://github.com/antirez/ds4

comment by sfifs · 2026-06-15 · in MAI-Code-1-Flash