Picovoice/picollm
View on GitHub ↗ ★ 319 Python
On-device LLM Inference Powered by X-Bit Quantization
3 mentions on Hacker News by 3 people: 1 story, 2 comments · first 2024-05-29 · latest 2025-04-06
20082026
-
…what I was looking for myself in a few minutes. https://youtu.be/6LHNbeDADA4?si=LCM2E48hVxmO6VG4 https://github.com/Picovoice/picollm PicoLLM is a way to run LLaMa 3 on Node, it will be great for my students. I bet you didn’t know much…
comment by EGreg · 2025-04-06 · in The Llama 4 herd -
▲ 15
On-Device LLM Inference Powered by X-Bit Quantizationstory by dynamix · 2024-05-29 · 0 comments
-
GitHub: https://github.com/Picovoice/picollm Platform: https://picovoice.ai/blog/picollm-local-llm-platform/ LLM Quantization Algorithm: https://picovoice.ai/blog/picollm-towards-optimal-llm-quanti... LLM Inference Engine:…
comment by kenarsa · 2024-05-29