cactus-compute/cactus
View on GitHub ↗ ★ 6,170 C++
Quantization, kernels, runtime and inference engine for mobiles, wearables, smart home and robots.
12 mentions on Hacker News by 8 people: 4 stories, 8 comments · first 2025-07-06 · latest 2026-08-16
20082026
-
…of your $250 chip and (2) for them to tailor their runtime and quantizations to such FPGA hardware? https://github.com/cactus-compute/cactus Also have you thought about using a Alveo V80? Still not crazy expensive and could fit bigger…
comment by gbxk · 2026-08-16 · in Show HN: A tiny LLM running at 21,000 tok/s on a $250 FPGA (Live Demo) -
…models like Whisper & Parakeet, benchmarks available in the attached GitHub repo. GitHub: https://github.com/cactus-compute/cactus We are keen to get your feedback, and star if feeling generous. Thanks a million
comment by cyann · 2026-02-26 · in Tell HN: YC companies scrape GitHub activity, send spam emails to users -
Can you clarify the following sentence: > We are open-source ( https://github.com/cactus-compute/cactus ). Cactus is free for hobbyists and personal projects, with a paid license required for commercial use. If it is open-source, one is…
comment by cientifico · 2025-09-18 · in Launch HN: Cactus (YC S25) – AI inference on smartphones -
Open source for the PR, then switching to non-open licensing is a cowardly, bullshit move. https://github.com/cactus-compute/cactus/commit/b1b5650d1132... Use open source and stick with it, or don't touch it at all, and tell any VC…
comment by observationist · 2025-09-18 · in Launch HN: Cactus (YC S25) – AI inference on smartphones -
…there is [2] [0] https://github.com/vercel/ai [1] https://github.com/ClickHouse/ai-sdk-cpp [2] https://github.com/cactus-compute/cactus
comment by pzo · 2025-07-22 · in Show HN: Any-LLM – Lightweight router to access any LLM Provider -
You don't need to guess: https://github.com/cactus-compute/cactus/tree/main/cpp
comment by liuliu · 2025-07-11 · in Show HN: Cactus – Ollama for Smartphones -
…DB and run Cactus embeddings for retrieval. Assuming you're using React Native, here's an example: https://github.com/cactus-compute/cactus/tree/main/react#emb... (Flutter works the same way) What are you building?
comment by rshemet · 2025-07-10 · in Show HN: Cactus – Ollama for Smartphones -
…of AI. Wanting to be able to run AI fully privately, and offline is the reason we created Cactus: https://github.com/cactus-compute/cactus Fully open-source, cross-platform & blazing-fast; lets you plug in private AI into any app on your…
comment by rshemet · 2025-07-06 · in Local-first software (2019)