Hacker GitHub

← All repos

toverainc/willow-inference-server

View on GitHub ↗ ★ 511 Python

Open source, local, and self-hosted highly optimized language inference server supporting ASR/STT, TTS, and LLM across WebRTC, REST, and WS

11 mentions on Hacker News by 2 people: 1 story, 10 comments · first 2023-05-23 · latest 2025-03-16

Mentions per year 11 mentions · busiest year 2023 (9)
20082026
AllStoriesComments NewestTopOldest
  1. …capable enough to have removed Amazon/Google anyways with HomeAssistant in combination with WIS ( https://github.com/toverainc/willow-inference-server ) and Willow autocorrect ( https://github.com/toverainc/willow-autocorrect ). Which gets…
    comment by horsawlarway · 2025-03-16 · in Amazon Is Discontinuing the "Do Not Send Voice Recordings" Feature on Echo
  2. …the Jetson line for many years and frequently struggled with all of these issues and more. [0] - https://github.com/toverainc/willow-inference-server/discuss...
    comment by kkielhofner · 2024-01-28 · in Brave Leo now uses Mixtral 8x7B as default
  3. …Alexa functionality but also far surpass it by leveraging the entire open source ecosystem. [0] - https://github.com/toverainc/willow-inference-server/discuss...
    comment by kkielhofner · 2023-11-02 · in Home Assistant 2023.11
  4. …has an API that can be used directly like OP project and supports all Whisper models, TTS, etc: https://github.com/toverainc/willow-inference-server The benchmarks are pretty incredible (largely thanks to ctranslate2).
    comment by kkielhofner · 2023-08-23 · in Whisper.api: Open-source, self-hosted speech-to-text with fast transcription
  5. …benchmark of 12x is from a very early access benchmark with some driver and other issues. [0] - https://github.com/toverainc/willow-inference-server/tree/wi...
    comment by kkielhofner · 2023-07-08 · in Train an AI model once and deploy on any cloud
  6. …I would mention. [0] - https://github.com/toverainc/willow [1] - https://wisng.tovera.io/rtc/ [2] - https://github.com/toverainc/willow-inference-server/tree/wi... [3] - https://github.com/toverainc/willow-inference-server [4] -…
    comment by kkielhofner · 2023-07-03 · in Show HN: Project S.A.T.U.R.D.A.Y. – open-source, self hosted, J.A.R.V.I.S.
  7. …a response makes pulling your phone out faster). [0] - https://github.com/toverainc/willow [1] - https://github.com/toverainc/willow-inference-server [2] - https://github.com/toverainc/willow-inference-server/tree/wi...
    comment by kkielhofner · 2023-06-21 · in vLLM: Easy, Fast, and Cheap LLM Serving with PagedAttention
  8. …is 500ms from end of speech to command executed. [0] - https://github.com/toverainc/willow [1] - https://github.com/toverainc/willow-inference-server
    comment by kkielhofner · 2023-06-07 · in GGML – AI at the Edge
  9. …The space needs some real competition somewhere. [0] - https://github.com/toverainc/willow [1] - https://github.com/toverainc/willow-inference-server
    comment by kkielhofner · 2023-06-06 · in Nvidia releases new AI chip with 480GB CPU RAM, 96GB GPU RAM
  10. …model settings on a per-request basis to balance quality vs latency depending on the task. [0] - https://github.com/toverainc/willow-inference-server#benchma...
    comment by kkielhofner · 2023-05-23 · in Show HN: Willow Inference Server: Optimized ASR/TTS/LLM for Willow/WebRTC/REST