Hacker GitHub

← All repos

oobabooga/textgen

View on GitHub ↗ ★ 47,729 Python

Open-source desktop app for local LLMs. Text, vision, tool-calling, OpenAI/Anthropic-compatible API. 100% private.

131 mentions on Hacker News by 98 people: 7 stories, 124 comments · first 2023-02-26 · latest 2025-10-26

Mentions per year 131 mentions · busiest year 2023 (108)
20082026
AllStoriesComments NewestTopOldest
  1. …alternatives for the post. Best thing you can really do is try them yourself with a tool like this https://github.com/oobabooga/text-generation-webui Benchmarks are one thing but I suspect if any were truly on par we would know about it.…
    comment by factoidforrest · 2023-03-25 · in OpenAI Vendor Lock-How OpenAI Went from Open Source to “Open Your Wallet”
  2. I'm running LLaMA-65B on a a2-ultragpu-1g instance at GCP with a 1xNVIDIA A100 80GB using this UI: https://github.com/oobabooga/text-generation-webui The good thing about this UI is that it supports both completion and chat-mode (+ is…
    comment by rvnx · 2023-03-23 · in OpenAI’s policies hinder reproducible research on language models
  3. most implementations do, like https://github.com/oobabooga/text-generation-webui this might be a hallucinated answer, due to the very small model size of 7b. try the 13b-4bit, it's much better!
    comment by underlines · 2023-03-22 · in Show HN: ChatLLaMA – A ChatGPT style chatbot for Facebook's LLaMA
  4. …and just tinkering with things on my own really helped. Through the FlexGen discord I've discovered https://github.com/oobabooga/text-generation-webui where I spent days just playing around with models. This got me into the huggingface…
    comment by lxe · 2023-03-22 · in Show HN: Finetune LLaMA-7B on commodity GPUs using your own text
  5. …webui tool which has managed to integrate almost all the hottest new feature forks fairly quickly: https://github.com/oobabooga/text-generation-webui I just helped someone recently get oobabooga running which has a --listen option to open…
    comment by glenneroo · 2023-03-15 · in GPT-4
  6. Another great option is https://github.com/oobabooga/text-generation-webui The 7B model will run without changes on a 3080. The 13B 4-bit model also runs on a 3080. This Reddit post has the instructions I followed:…
    comment by eulers_secret · 2023-03-13 · in Alpaca: A strong open-source instruction-following model
  7. Wow, yeah that's a VERY active project: https://github.com/oobabooga/text-generation-webui/graphs/co... - only started Dec 18, 2022 and already 22 contributors and 806 commits!
    comment by simonw · 2023-03-13 · in Alpaca: A strong open-source instruction-following model
  8. …(seriously, most of the cutting edge with 4-bit GPTQ etc. has been closely tied to the project). > https://github.com/oobabooga/text-generation-webui/
    comment by loufe · 2023-03-13 · in Alpaca: A strong open-source instruction-following model
  9. …3060 card. I have 4 bit 13B parameter LLaMA running on it currently, following these instructions - https://github.com/oobabooga/text-generation-webui/wiki/LLaM... . They don't have 30B or 65B ready yet. Might try other methods to do 30B,…
    comment by Ologn · 2023-03-12 · in Using LLaMA with M1 Mac and Python 3.11
  10. https://github.com/oobabooga/text-generation-webui/wiki/LLaM... Here you go. You'll need a Nvidia GPU with at least 8 GB VRAM though.
    comment by zirgs · 2023-03-12 · in Using LLaMA with M1 Mac and Python 3.11
  11. https://github.com/oobabooga/text-generation-webui/ has been running 4bit LLaMA on Radeon hardware with faster speeds and lower memory requirements than this LLaMA.cpp for the past week!
    comment by MacsHeadroom · 2023-03-12 · in Running LLaMA 7B on a 64GB M2 MacBook Pro with Llama.cpp
  12. …3090, RTX 4090, RTX 6000, Tesla V100, A100 40GB, A40, RTX A6000, RTX 8000, Titan Ada Mostly using https://github.com/oobabooga/text-generation-webui/ , the AUTOMATIC1111 of textgen.
    comment by MacsHeadroom · 2023-03-12 · in Large language models are having their Stable Diffusion moment
  13. Massively diverse community working on the AUTOMATIC1111 of textgen at https://github.com/oobabooga/text-generation-webui/ Ooba's textgen webui runs LLaMA 4bit on 8GB VRAM consumer video cards. KoboldAI (…
    comment by MacsHeadroom · 2023-03-12 · in Large language models are having their Stable Diffusion moment
  14. …have been running LLaMA in 4bit quickly on cheap hardware with a simple GUI for over a week using https://github.com/oobabooga/text-generation-webui Just not on Macs. (that repo does not support Apple Silicon)
    comment by MacsHeadroom · 2023-03-12 · in Llama.cpp: Port of Facebook's LLaMA model in C/C++, with Apple Silicon support
  15. …I also wrote a guide how to get the bitsandbytes library working on windows: https://github.com/oobabooga/text-generation-webui/issues/14...
    comment by underlines · 2023-03-11 · in Llama.cpp: Port of Facebook's LLaMA model in C/C++, with Apple Silicon support
  16. With a recent PR, text-generation-webui makes this very easy to use https://github.com/oobabooga/text-generation-webui/pull/206
    comment by nabakin · 2023-03-11 · in Llama.cpp: Port of Facebook's LLaMA model in C/C++, with Apple Silicon support
  17. I've gotten great q&a / conversational results from llama 7b and 13b using https://github.com/oobabooga/text-generation-webui . ], [ "How does the Schrodinger's cat thought experiment relate to the principles of quantum mechanics?", "In…
    comment by lxe · 2023-03-10 · in Llama.cpp: Port of Facebook's LLaMA model in C/C++, with Apple Silicon support
  18. …[0] https://github.com/qwopqwop200/GPTQ-for-LLaMa [1] https://arxiv.org/abs/2210.17323 [2] https://github.com/oobabooga/text-generation-webui/issues/14... For LLaMA set-up instructions, including GPTQ 4bit, refer to this wiki article:…
    comment by MacsHeadroom · 2023-03-10 · in LLaMA Text-Generation-Webui
  19. A web UI for running Large Language Models https://github.com/oobabooga/text-generation-webui
    comment by amrb · 2023-03-10 · in Ask HN: Is there a good alternative for a local ChatGPT
  20. …4bit needs 36 GB of VRAM, but far exceeds GPT-3's capabilities and even takes on PaLM 540B. See: https://github.com/oobabooga/text-generation-webui/wiki/LLaM... for 4bit setup instructions See also: The case for 4-bit precision, which…
    comment by MacsHeadroom · 2023-03-10 · in Visual ChatGPT
  21. …potential (from what I understand). [1] https://github.com/huggingface/transformers/pull/21955 [2] https://github.com/oobabooga/text-generation-webui/commit/90...
    comment by loufe · 2023-03-05 · in Show HN: Llama-dl – high-speed download of LLaMA, Facebook's 65B GPT model
  22. In int8 7B needs only 9GB of VRAM and 13B needs only 20GB on a single GPU. https://github.com/oobabooga/text-generation-webui/issues/14...
    comment by MacsHeadroom · 2023-03-05 · in Facebook LLAMA is being openly distributed via torrents
  23. …You can run the second smallest one (better than GPT-3 175B) on 24GB of vram, ie LLaMA-13B. https://github.com/oobabooga/text-generation-webui/issues/14...
    comment by MacsHeadroom · 2023-03-05 · in Facebook LLAMA is being openly distributed via torrents
  24. …today and came around plenty of resource but hard to make any sense of it. I had this working https://github.com/oobabooga/text-generation-webui and instead of it being able to answer question, it revolves around the concept of generating…
    comment by mattfrommars · 2023-02-26 · in Using GPT3, Supabase and Pinecone to automate a personalized marketing campaign