Hacker GitHub

← All repos

huggingface/text-generation-inference

View on GitHub ↗ ★ 10,883 Python archived

Large Language Model Text Generation Inference

13 mentions on Hacker News by 12 people: 4 stories, 9 comments · first 2023-02-04 · latest 2024-04-12

Mentions per year 13 mentions · busiest year 2023 (9)
20082026
AllStoriesComments NewestTopOldest
  1. …Source anymore, but they have reverted the license back to Apache 2.0 for the new version TGI v2.0: https://github.com/huggingface/text-generation-inference/rel... Good news!
    comment by dloss · 2024-04-12 · in Zephyr 141B, a Mixtral 8x22B fine-tune, is now available in Hugging Chat
  2. …https://huggingface.co/datasets/argilla/distilabel-capybara-... - Open-source inference engine: https://github.com/huggingface/text-generation-inference - Open-source UI code https://github.com/huggingface/chat-ui Have fun!
    comment by osanseviero · 2024-04-12 · in Zephyr 141B, a Mixtral 8x22B fine-tune, is now available in Hugging Chat
  3. ▲ 3
    Hugging Face reverts the license back to Apache 2.0
    story by vmatsiiako · 2024-04-09 · 0 comments
  4. ▲ 19
    HuggingFace text-generation-inference is reverting to Apache 2.0 License
    story by Thicken2320 · 2024-04-08 · 1 comment
  5. …large language mode Language Server (is this making sense yet) 3) HuggingFace inference framework. https://github.com/huggingface/text-generation-inference At least when I tested you couldn't use something like llama.cpp or exllama with…
    comment by mcbuilder · 2023-10-19 · in LocalPilot: Open-source GitHub Copilot on your MacBook
  6. A simple microservice would be https://github.com/huggingface/text-generation-inference . Works flawlessly in Docker on my Windows machine, which is quite shocking. Supports Mistral as well as everything else. Biggest downside is that…
    comment by LoganDark · 2023-10-11 · in Mistral 7B
  7. …if done today. Would be very interesting to see more recent benchmarks if anyone has done any! [1] https://github.com/huggingface/text-generation-inference/iss...
    comment by okalldal · 2023-08-15 · in Continuous batching to increase LLM inference throughput and reduce p50 latency
  8. …license by a non-hyperscaler user https://github.com/Preemo-Inc/text-generation-inference https://github.com/huggingface/text-generation-inference/iss...
    comment by mlinksva · 2023-07-29 · in HuggingFace Text Generation License No Longer Open-Source
  9. ▲ 84
    HuggingFace Text Generation License No Longer Open-Source
    story by bratao · 2023-07-29 · 55 comments
  10. ▲ 6
    HuggingFace Text Generation Library License Changed from Apache 2 to Hfoil
    story by bratao · 2023-07-28 · 3 comments
  11. …comprehensive... If you have a linux machine with GPUs, i'd just use hugging face inference ( https://github.com/huggingface/text-generation-inference ). And I am sure there are other things that could be covered.
    comment by andreyk · 2023-07-25 · in Guide to running Llama 2 locally
  12. …inference, the HuggingFace cofounder, Thom Wolf recommends their text-generation-inference library https://github.com/huggingface/text-generation-inference https://twitter.com/Thom_Wolf/status/1664613366751526914
    comment by convexstrictly · 2023-06-02 · in Falcon 40B LLM (which beats Llama) now Apache 2.0
  13. …< https://hub.docker.com/layers/ykilcher/text-generation-infer... > it seems to match up with https://github.com/huggingface/text-generation-inference/blo...
    comment by mdaniel · 2023-02-04 · in Open Assistant: Conversational AI for Everyone