Hacker GitHub

← All repos

oobabooga/textgen

View on GitHub ↗ ★ 47,729 Python

Open-source desktop app for local LLMs. Text, vision, tool-calling, OpenAI/Anthropic-compatible API. 100% private.

131 mentions on Hacker News by 98 people: 7 stories, 124 comments · first 2023-02-26 · latest 2025-10-26

Mentions per year 131 mentions · busiest year 2023 (108)
20082026
AllStoriesComments NewestTopOldest
  1. …AI histira was a little more rampant but still. tl;dr my recommendation for your pdf project is run https://github.com/oobabooga/text-generation-webui . if your can get a 30 series GPU in your company Then run a 13B 4bit model that can…
    comment by rig666 · 2023-08-16 · in Why host your own LLM?
  2. …little messy and cobbled together I run oobabooga's API on docker with a 13B 4bit quantized model. https://github.com/oobabooga/text-generation-webui We use GTX 3060s because there the best bang for there buck in terms of VRAM. Our current…
    comment by rig666 · 2023-08-15 · in Why host your own LLM?
  3. …chatGPT (3.5) for general tasks, but can be configured to be quite good at very specific things. https://github.com/oobabooga/text-generation-webui https://github.com/go-skynet/LocalAI https://github.com/gitmylo/audio-webui
    comment by smcleod · 2023-08-14 · in ChatGPT costs 700k/day to run; may go bankrupt in 2024
  4. …nowhere near the level of GPT-4. It's close enough to GPT 3.5 though, you should try it out! [1]: https://github.com/oobabooga/text-generation-webui
    comment by pocketarc · 2023-08-03 · in Vicuna v1.5 series, featuring 4K and 16K context, based on Llama 2
  5. I've tried running some using https://github.com/oobabooga/text-generation-webui on my arch box with an AMD gpu and it worked well and was really handy, until I tried yesterday and I get some python error no matter what I do, with the same…
    comment by tapland · 2023-07-26 · in Show HN: Experiment with Hugging Face models in a single notebook interface
  6. …it will be worth it. I would highly recommend looking into the text-generation-webui project ( https://github.com/oobabooga/text-generation-webui ). It has a one-click installer and very comprehensive guides for getting models running…
    comment by mattstir · 2023-07-26 · in Guide to running Llama 2 locally
  7. …got same answers. I even got better answers with ooba, since the demo cuts text early Ooobabooga: https://github.com/oobabooga/text-generation-webui Model: TheBloke_Llama-2-70B-chat-GPTQ from https://huggingface.co/TheBloke/Llama-2-70B-chat…
    comment by eurekin · 2023-07-25 · in Guide to running Llama 2 locally
  8. …as always, is the oogabooga text generation webUI, which supports all of the relevant backends: https://github.com/oobabooga/text-generation-webui
    comment by Der_Einzige · 2023-07-25 · in Guide to running Llama 2 locally
  9. …in case people come here because the title sounds interesting, check out text-generation-webui ( https://github.com/oobabooga/text-generation-webui ). It's a better solution to local LLMs than running in a Jupyter Notebook.
    comment by heyitsguay · 2023-07-24 · in Running Open Source LLMs in Jupyter Python Notebook as an Alternative to ChatGPT
  10. Simon's article didn't show local usage. Use one of the one-click installers linked in the README of https://github.com/oobabooga/text-generation-webui and you're set. Note that just in case you have the hardware necessary to run the…
    comment by Tepix · 2023-07-19 · in Accessing Llama 2 from the command-line with the LLM-replicate plugin
  11. The real gold standard is https://github.com/oobabooga/text-generation-webui Which includes the llama.cpp backend, and a lot more. Unfortunately, despite claiming to be the "Automatic1111" of text generation, it doesn't support any of the…
    comment by Der_Einzige · 2023-07-19 · in Accessing Llama 2 from the command-line with the LLM-replicate plugin
  12. https://github.com/oobabooga/text-generation-webui https://github.com/bentoml/OpenLLM https://www.reddit.com/r/LocalLLaMA/top/?t=month
    comment by JimmyRuska · 2023-07-08 · in PdfGptIndexer: Indexing and searching PDF text data using GPT-2 and FAISS
  13. …(falcon) For interactive work (art/chat/research/playing around), things like: https://github.com/oobabooga/text-generation-webui/blob/main... (llama) (Also - they just added a decent chat server built into llama.cpp the project) https:/
    comment by horsawlarway · 2023-07-07 · in ChatGPT loses users for first time, shaking faith in AI revolution
  14. There's the OpenedAI-API extension for text-generation-webui: https://github.com/oobabooga/text-generation-webui/tree/main...
    comment by fortydegrees · 2023-06-19 · in OpenLLM
  15. …for each of the GGML models, which are best if you want to host using CPU/RAM (no video card). [0] https://github.com/oobabooga/text-generation-webui/blob/main... [1] https://huggingface.co/TheBloke
    comment by SparkyMcUnicorn · 2023-06-13 · in Obsidian-Copilot: A Prototype Assistant for Writing and Thinking
  16. …There's a web gui for llama.cpp that is really straightforward to set up for huggingface models: https://github.com/oobabooga/text-generation-webui
    comment by moffkalast · 2023-06-04 · in XB, or eXtreme Bullshitting: a less misleading name for LLMs
  17. …it is to use text-generation-webui[1] and 33B 4-bit quantized GGML model from TheBloke[2]. [1] https://github.com/oobabooga/text-generation-webui [2] https://huggingface.co/TheBloke/guanaco-33B-GGML
    comment by coolspot · 2023-05-31 · in Ask HN: What's the best self hosted/local alternative to GPT-4?
  18. https://github.com/oobabooga/text-generation-webui/ Works on all platforms, but runs much better on Linux. Running this in Docker on my 2080Ti, can barely fit 13B-4bit models into 11G of VRAM, but it works fine, produces around 10-15…
    comment by weystrom · 2023-05-31 · in Ask HN: What's the best self hosted/local alternative to GPT-4?
  19. …I haven't had much of my time available for this recently. My recommendation would be to start with https://github.com/oobabooga/text-generation-webui You will find almost everything you need to know there and on 4chan.org/g/catalog -…
    comment by hanselot · 2023-05-31 · in Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?
  20. …super confusing is people on r/LocalLLaMA often just refer too it as "ooba". The actual repo is https://github.com/oobabooga/text-generation-webui and you notice that oobabooga is actually the username, but a lot of non-technical people…
    comment by barking_biscuit · 2023-05-26 · in Gorilla: Large Language Model connected with massive APIs
  21. If you're running the text-generation-webui ( https://github.com/oobabooga/text-generation-webui ) it has the ability to train LoRAs. It'll require a beefy GPU but I've seen some fun examples like someone training a LoRA on Skyrim books.
    comment by a5huynh · 2023-05-25 · in How to Finetune GPT-Like Large Language Models on a Custom Dataset
  22. These aren't for ChatGPT, but work on LLaMA, Vicuna, etc. https://github.com/oobabooga/text-generation-webui/blob/main... https://github.com/zetavg/LLaMA-LoRA-Tuner https://github.com/h2oai/h2o-llmstudio https://github.com/rhulha/lora
    comment by SparkyMcUnicorn · 2023-05-25 · in How to Finetune GPT-Like Large Language Models on a Custom Dataset
  23. …supporting ooba's api which would give you a lot of support for different things really quickly. https://github.com/oobabooga/text-generation-webui/
    comment by theaiquestion · 2023-05-18 · in Show HN: Smallville – Create generative agents for simulations and games
  24. I'm not sure how this extension compares with https://github.com/oobabooga/text-generation-webui/tree/main...
    comment by rahimnathwani · 2023-05-15 · in Long-term memory (LTM) extension for oobabooga's Text Generation Web UI
  25. …good place to dig for prompt structures may be the 'text-generation-webui' commit log. For example https://github.com/oobabooga/text-generation-webui/commit/33...
    comment by rain1 · 2023-05-14 · in Run Llama 13B with a 6GB graphics card
  26. …it an AI girlfriend isnt even hiding it. Secondly if you want an AI girlfriend just use oobabooga ( https://github.com/oobabooga/text-generation-webui ) and prompt your own AI girlfriend. I feel no remorse for those that pay a dollar per…
    comment by rig777 · 2023-05-13 · in An AI Girlfriend made $72K in 1 week
  27. …get an acceptable performance without the headaches right now, if you can afford the price. Install https://github.com/oobabooga/text-generation-webui , update pytorch and llamacpp-python, and you should be able to run pretty much all…
    comment by execveat · 2023-05-08 · in RasaGPT: First headless LLM chatbot built on top of Rasa, Langchain and FastAPI
  28. …current generation of open models (OpenAssistant w/ 30B LLaMa) [0]: https://mlc.ai/web-llm/ [1]: https://github.com/oobabooga/text-generation-webui [2]: https://huggingface.co/chat
    comment by infinityio · 2023-05-06 · in America Forgot About IBM Watson. Is ChatGPT Next?
  29. …until they reach a good loss(the model works), that's why quality data is important. You might try https://github.com/oobabooga/text-generation-webui they have a pretty easy setup config. Again, you'll need a lot of RAM and a good CPU for…
    comment by examplary_cable · 2023-05-06 · in Releasing 3B and 7B RedPajama
  30. Run text-generation-webui with llama.cpp: https://github.com/oobabooga/text-generation-webui
    comment by sjy · 2023-05-05 · in Google “We have no moat, and neither does OpenAI”
  31. …with checkpoints, various extensions including multimodal and experimental long-term memory etc. https://github.com/oobabooga/text-generation-webui
    comment by int_19h · 2023-05-04 · in Google “We have no moat, and neither does OpenAI”
  32. Not for training with the webui: https://github.com/oobabooga/text-generation-webui/issues/11... It does seem to work using alpca-lora directly, though.
    comment by eulers_secret · 2023-05-04 · in Google “We have no moat, and neither does OpenAI”
  33. This really ought to mention https://github.com/oobabooga/text-generation-webui , which was the first popular UI for LLaMA, and remains one for anyone who runs it on GPU. It is also where GPTQ 4-bit quantization was first enabled in a…
    comment by int_19h · 2023-05-04 · in Google “We have no moat, and neither does OpenAI”
  34. …could be used by an extension your IDE uses. But I'm not sure if an extension like that exists. [0] https://github.com/oobabooga/text-generation-webui
    comment by jacobrussell · 2023-05-03 · in Replit's new Code LLM: Open Source, 77% smaller than Codex, trained in 1 week
  35. …The Oobabooga web interface aims to become the standard interface for chat models: https://github.com/oobabooga/text-generation-webui I don't see any indication that OpenLLaMa will run on either of those without modification. But one of…
    comment by loudmax · 2023-05-03 · in OpenLLaMA: An Open Reproduction of LLaMA
  36. …unversioned and mere weeks after being written simply doesn't work as a direct consequence . E.g: https://github.com/oobabooga/text-generation-webui/blob/ee68... Take this line: pip3 install torch torchvision torchaudio Which version of…
    comment by jiggawatts · 2023-04-30 · in A brief history of LLaMA models
  37. https://github.com/oobabooga/text-generation-webui/blob/main... You need to enable the extensions. I only did voice out locally with silero_tts, it also supports voice out with eleven labs api. Voice input is via whisper tts.
    comment by IronWolve · 2023-04-25 · in QuiLLMan: Voice chat with Vicuna-13B
  38. …it has voice in/out, graphics in/out, memory plugin, api, plugins, etc, all running locally. https://github.com/oobabooga/text-generation-webui
    comment by IronWolve · 2023-04-25 · in QuiLLMan: Voice chat with Vicuna-13B
  39. Ah. Well you can build a dockerfile (with the 4 bit triton repo) locally here, then deploy it: https://github.com/oobabooga/text-generation-webui#alternati... I havent explored beyond that. Actually there are tons of quantizatons on…
    comment by brucethemoose2 · 2023-04-23 · in Ask HN: Are there any ready-to-use Docker images for running LLMs locally?
  40. …llama is messy because there are essentially 3 variants. - 4 bit cuda - 4 bit triton - 4 bit cpu https://github.com/oobabooga/text-generation-webui/blob/main... Models should be quantized specifically for each, and both branches are under…
    comment by brucethemoose2 · 2023-04-23 · in Ask HN: Are there any ready-to-use Docker images for running LLMs locally?
  41. This plus the the vicuna models is probably as close as you can get in the near term. https://github.com/oobabooga/text-generation-webui
    comment by ospray · 2023-04-18 · in GPT4All Chat – Locally-running AI chat application powered by the GPT4All-J
  42. ▲ 5
    Show HN: Example POC to run the Dolly V2 12B model in 8bit mode
    story by freeqaz · 2023-04-14 · 0 comments
  43. ▲ 3
    How to stop Vicuna talking to itself
    story by syntaxing · 2023-04-10 · 0 comments
  44. …as well, but maybe I'll see if the extra cycles helps). I'm giving 30B a try on my GPU now with https://github.com/oobabooga/text-generation-webui/wiki/LLaM... and if it's not good then will give layer offloading with 65B a try and see if…
    comment by lhl · 2023-04-08 · in State-of-the-Art Chatbot, Vicuna-7B, now runs on MacBook with GPU acceleration
  45. The default loader doesn't seem to let you load quantized models but if you use something like https://github.com/oobabooga/text-generation-webui you can 1) use the model with `--load-in-8bit` which halves the memory (runs on my 24GB…
    comment by lhl · 2023-04-05 · in State-of-the-art open-source chatbot, Vicuna-13B, just released model weights
  46. …python bindings for llama.cpp and the text-generation-webui already uses them for local inference. https://github.com/oobabooga/text-generation-webui/wiki/llam... "pip install llamacpp" or https://github.com/thomasantony/llamacpp-python
    comment by MacsHeadroom · 2023-04-04 · in An LLM playground you can run on your laptop
  47. As an alternative for purely local LLMs, I've been having fun with this setup: https://github.com/oobabooga/text-generation-webui
    comment by a5huynh · 2023-04-04 · in An LLM playground you can run on your laptop
  48. https://github.com/oobabooga/text-generation-webui
    comment by enlyth · 2023-03-31 · in Llama.cpp 30B runs with only 6GB of RAM now
  49. …the discussion of that second paper here is interesting, though it doesn't have its own benchmarks: https://github.com/oobabooga/text-generation-webui/issues/17...
    comment by corvec · 2023-03-31 · in Llama.cpp 30B runs with only 6GB of RAM now
  50. …the same result. If you want to run larger Alpaca models on a low VRAM GPU, try FlexGen. I think https://github.com/oobabooga/text-generation-webui/ is one of the easier ways to get that going.
    comment by sp332 · 2023-03-31 · in Llama.cpp 30B runs with only 6GB of RAM now