Hacker GitHub

← All repos

OpenNMT/CTranslate2

View on GitHub ↗ ★ 4,729 C++

Fast inference engine for Transformer models

12 mentions on Hacker News by 7 people: 1 story, 11 comments · first 2023-02-20 · latest 2025-01-17

Mentions per year 12 mentions · busiest year 2023 (11)
20082026
AllStoriesComments NewestTopOldest
  1. …a PR in ctranslate2 implementing the same feature so that it could be exposed in faster-whisper: https://github.com/OpenNMT/CTranslate2/pull/1789
    comment by woodson · 2025-01-17 · in Brood War Korean Translations
  2. …that enables a lot. They should be up front and center and get the credit they deserve. [0] - https://github.com/OpenNMT/CTranslate2
    comment by kkielhofner · 2023-11-01 · in Distil-Whisper: distilled version of Whisper that is 6 times faster, 49% smaller
  3. I have the same plans for ctranslate2[0] and Willow Inference Server[1]! [0] - https://github.com/OpenNMT/CTranslate2 [1] - https://heywillow.io/components/willow-inference-server/
    comment by kkielhofner · 2023-11-01 · in Distil-Whisper: distilled version of Whisper that is 6 times faster, 49% smaller
  4. …but we see this as being one of several fundamental issues in supporting AMD GPGPU hardware. [0] - https://github.com/OpenNMT/CTranslate2 [1] - https://github.com/OpenNMT/CTranslate2/issues/1072
    comment by kkielhofner · 2023-10-09 · in A Raspberry Pi 5 is better than two Pi 4S
  5. …in CTranslate2 where they clearly state no-one is working to get ROCm supported in the library. ( https://github.com/OpenNMT/CTranslate2/issues/1072#issuecomm ... ) I don't understand what you're saying here. It (along with the other…
    comment by kkielhofner · 2023-10-07 · in AMD may get across the CUDA moat
  6. …in CTranslate2 where they clearly state no-one is working to get ROCm supported in the library. ( https://github.com/OpenNMT/CTranslate2/issues/1072#issuecomm... )
    comment by errnoh · 2023-10-07 · in AMD may get across the CUDA moat
  7. …it presents. References: https://github.com/InternLM/lmdeploy https://github.com/vllm-project/vllm https://github.com/OpenNMT/CTranslate2 You know what’s missing from all of these and many more like them? Support for ROCm. This is all…
    comment by kkielhofner · 2023-10-07 · in AMD may get across the CUDA moat
  8. …I’m only more and more skeptical that real competition in the space will come from AMD. [0] - https://github.com/OpenNMT/CTranslate2
    comment by kkielhofner · 2023-10-07 · in AMD may get across the CUDA moat
  9. …In such case, what you could do is to cache at least the system prompt. This is also possible with https://github.com/OpenNMT/CTranslate2/blob/2203ad5c8baf878a...
    comment by Palmik · 2023-10-04 · in StreamingLLM: Efficient streaming technique enable infinite sequence lengths
  10. …same model but using another backend, which is specifically optimized for inference workloads. [1] https://github.com/OpenNMT/CTranslate2
    comment by guillaumekln · 2023-07-21 · in Faster Whisper Transcription with CTranslate2
  11. …like PyTorch. There are C++ implementations but they optimize on different aspects. For example: https://github.com/OpenNMT/CTranslate2/
    comment by albertzeyer · 2023-02-20 · in Running large language models like ChatGPT on a single GPU