oobabooga/textgen on Hacker News

…chatGPT (3.5) for general tasks, but can be configured to be quite good at very specific things. https://github.com/oobabooga/text-generation-webui https://github.com/go-skynet/LocalAI https://github.com/gitmylo/audio-webui

comment by smcleod · 2023-08-14 · in ChatGPT costs 700k/day to run; may go bankrupt in 2024

…nowhere near the level of GPT-4. It's close enough to GPT 3.5 though, you should try it out! [1]: https://github.com/oobabooga/text-generation-webui

comment by pocketarc · 2023-08-03 · in Vicuna v1.5 series, featuring 4K and 16K context, based on Llama 2

I've tried running some using https://github.com/oobabooga/text-generation-webui on my arch box with an AMD gpu and it worked well and was really handy, until I tried yesterday and I get some python error no matter what I do, with the same…

comment by tapland · 2023-07-26 · in Show HN: Experiment with Hugging Face models in a single notebook interface

…it will be worth it. I would highly recommend looking into the text-generation-webui project ( https://github.com/oobabooga/text-generation-webui ). It has a one-click installer and very comprehensive guides for getting models running…

comment by mattstir · 2023-07-26 · in Guide to running Llama 2 locally

…got same answers. I even got better answers with ooba, since the demo cuts text early Ooobabooga: https://github.com/oobabooga/text-generation-webui Model: TheBloke_Llama-2-70B-chat-GPTQ from https://huggingface.co/TheBloke/Llama-2-70B-chat…

comment by eurekin · 2023-07-25 · in Guide to running Llama 2 locally

…as always, is the oogabooga text generation webUI, which supports all of the relevant backends: https://github.com/oobabooga/text-generation-webui

comment by Der_Einzige · 2023-07-25 · in Guide to running Llama 2 locally

…in case people come here because the title sounds interesting, check out text-generation-webui ( https://github.com/oobabooga/text-generation-webui ). It's a better solution to local LLMs than running in a Jupyter Notebook.

comment by heyitsguay · 2023-07-24 · in Running Open Source LLMs in Jupyter Python Notebook as an Alternative to ChatGPT

Simon's article didn't show local usage. Use one of the one-click installers linked in the README of https://github.com/oobabooga/text-generation-webui and you're set. Note that just in case you have the hardware necessary to run the…

comment by Tepix · 2023-07-19 · in Accessing Llama 2 from the command-line with the LLM-replicate plugin

The real gold standard is https://github.com/oobabooga/text-generation-webui Which includes the llama.cpp backend, and a lot more. Unfortunately, despite claiming to be the "Automatic1111" of text generation, it doesn't support any of the…

comment by Der_Einzige · 2023-07-19 · in Accessing Llama 2 from the command-line with the LLM-replicate plugin

https://github.com/oobabooga/text-generation-webui https://github.com/bentoml/OpenLLM https://www.reddit.com/r/LocalLLaMA/top/?t=month

comment by JimmyRuska · 2023-07-08 · in PdfGptIndexer: Indexing and searching PDF text data using GPT-2 and FAISS

…(falcon) For interactive work (art/chat/research/playing around), things like: https://github.com/oobabooga/text-generation-webui/blob/main... (llama) (Also - they just added a decent chat server built into llama.cpp the project) https:/

comment by horsawlarway · 2023-07-07 · in ChatGPT loses users for first time, shaking faith in AI revolution

There's the OpenedAI-API extension for text-generation-webui: https://github.com/oobabooga/text-generation-webui/tree/main...

comment by fortydegrees · 2023-06-19 · in OpenLLM

…for each of the GGML models, which are best if you want to host using CPU/RAM (no video card). [0] https://github.com/oobabooga/text-generation-webui/blob/main... [1] https://huggingface.co/TheBloke

comment by SparkyMcUnicorn · 2023-06-13 · in Obsidian-Copilot: A Prototype Assistant for Writing and Thinking

…There's a web gui for llama.cpp that is really straightforward to set up for huggingface models: https://github.com/oobabooga/text-generation-webui

comment by moffkalast · 2023-06-04 · in XB, or eXtreme Bullshitting: a less misleading name for LLMs

…it is to use text-generation-webui[1] and 33B 4-bit quantized GGML model from TheBloke[2]. [1] https://github.com/oobabooga/text-generation-webui [2] https://huggingface.co/TheBloke/guanaco-33B-GGML

comment by coolspot · 2023-05-31 · in Ask HN: What's the best self hosted/local alternative to GPT-4?

https://github.com/oobabooga/text-generation-webui/ Works on all platforms, but runs much better on Linux. Running this in Docker on my 2080Ti, can barely fit 13B-4bit models into 11G of VRAM, but it works fine, produces around 10-15…

comment by weystrom · 2023-05-31 · in Ask HN: What's the best self hosted/local alternative to GPT-4?

…I haven't had much of my time available for this recently. My recommendation would be to start with https://github.com/oobabooga/text-generation-webui You will find almost everything you need to know there and on 4chan.org/g/catalog -…

comment by hanselot · 2023-05-31 · in Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

…super confusing is people on r/LocalLLaMA often just refer too it as "ooba". The actual repo is https://github.com/oobabooga/text-generation-webui and you notice that oobabooga is actually the username, but a lot of non-technical people…

comment by barking_biscuit · 2023-05-26 · in Gorilla: Large Language Model connected with massive APIs

If you're running the text-generation-webui ( https://github.com/oobabooga/text-generation-webui ) it has the ability to train LoRAs. It'll require a beefy GPU but I've seen some fun examples like someone training a LoRA on Skyrim books.

comment by a5huynh · 2023-05-25 · in How to Finetune GPT-Like Large Language Models on a Custom Dataset

These aren't for ChatGPT, but work on LLaMA, Vicuna, etc. https://github.com/oobabooga/text-generation-webui/blob/main... https://github.com/zetavg/LLaMA-LoRA-Tuner https://github.com/h2oai/h2o-llmstudio https://github.com/rhulha/lora

comment by SparkyMcUnicorn · 2023-05-25 · in How to Finetune GPT-Like Large Language Models on a Custom Dataset

…supporting ooba's api which would give you a lot of support for different things really quickly. https://github.com/oobabooga/text-generation-webui/

comment by theaiquestion · 2023-05-18 · in Show HN: Smallville – Create generative agents for simulations and games

I'm not sure how this extension compares with https://github.com/oobabooga/text-generation-webui/tree/main...

comment by rahimnathwani · 2023-05-15 · in Long-term memory (LTM) extension for oobabooga's Text Generation Web UI

…good place to dig for prompt structures may be the 'text-generation-webui' commit log. For example https://github.com/oobabooga/text-generation-webui/commit/33...

comment by rain1 · 2023-05-14 · in Run Llama 13B with a 6GB graphics card

…it an AI girlfriend isnt even hiding it. Secondly if you want an AI girlfriend just use oobabooga ( https://github.com/oobabooga/text-generation-webui ) and prompt your own AI girlfriend. I feel no remorse for those that pay a dollar per…

comment by rig777 · 2023-05-13 · in An AI Girlfriend made $72K in 1 week

…get an acceptable performance without the headaches right now, if you can afford the price. Install https://github.com/oobabooga/text-generation-webui , update pytorch and llamacpp-python, and you should be able to run pretty much all…

comment by execveat · 2023-05-08 · in RasaGPT: First headless LLM chatbot built on top of Rasa, Langchain and FastAPI

…current generation of open models (OpenAssistant w/ 30B LLaMa) [0]: https://mlc.ai/web-llm/ [1]: https://github.com/oobabooga/text-generation-webui [2]: https://huggingface.co/chat

comment by infinityio · 2023-05-06 · in America Forgot About IBM Watson. Is ChatGPT Next?

…until they reach a good loss(the model works), that's why quality data is important. You might try https://github.com/oobabooga/text-generation-webui they have a pretty easy setup config. Again, you'll need a lot of RAM and a good CPU for…

comment by examplary_cable · 2023-05-06 · in Releasing 3B and 7B RedPajama

Run text-generation-webui with llama.cpp: https://github.com/oobabooga/text-generation-webui

comment by sjy · 2023-05-05 · in Google “We have no moat, and neither does OpenAI”

…with checkpoints, various extensions including multimodal and experimental long-term memory etc. https://github.com/oobabooga/text-generation-webui

comment by int_19h · 2023-05-04 · in Google “We have no moat, and neither does OpenAI”

Not for training with the webui: https://github.com/oobabooga/text-generation-webui/issues/11... It does seem to work using alpca-lora directly, though.

comment by eulers_secret · 2023-05-04 · in Google “We have no moat, and neither does OpenAI”

This really ought to mention https://github.com/oobabooga/text-generation-webui , which was the first popular UI for LLaMA, and remains one for anyone who runs it on GPU. It is also where GPTQ 4-bit quantization was first enabled in a…

comment by int_19h · 2023-05-04 · in Google “We have no moat, and neither does OpenAI”

…could be used by an extension your IDE uses. But I'm not sure if an extension like that exists. [0] https://github.com/oobabooga/text-generation-webui

comment by jacobrussell · 2023-05-03 · in Replit's new Code LLM: Open Source, 77% smaller than Codex, trained in 1 week

…The Oobabooga web interface aims to become the standard interface for chat models: https://github.com/oobabooga/text-generation-webui I don't see any indication that OpenLLaMa will run on either of those without modification. But one of…

comment by loudmax · 2023-05-03 · in OpenLLaMA: An Open Reproduction of LLaMA

…unversioned and mere weeks after being written simply doesn't work as a direct consequence . E.g: https://github.com/oobabooga/text-generation-webui/blob/ee68... Take this line: pip3 install torch torchvision torchaudio Which version of…

comment by jiggawatts · 2023-04-30 · in A brief history of LLaMA models

https://github.com/oobabooga/text-generation-webui/blob/main... You need to enable the extensions. I only did voice out locally with silero_tts, it also supports voice out with eleven labs api. Voice input is via whisper tts.

comment by IronWolve · 2023-04-25 · in QuiLLMan: Voice chat with Vicuna-13B

…it has voice in/out, graphics in/out, memory plugin, api, plugins, etc, all running locally. https://github.com/oobabooga/text-generation-webui

comment by IronWolve · 2023-04-25 · in QuiLLMan: Voice chat with Vicuna-13B

Ah. Well you can build a dockerfile (with the 4 bit triton repo) locally here, then deploy it: https://github.com/oobabooga/text-generation-webui#alternati... I havent explored beyond that. Actually there are tons of quantizatons on…

comment by brucethemoose2 · 2023-04-23 · in Ask HN: Are there any ready-to-use Docker images for running LLMs locally?

…llama is messy because there are essentially 3 variants. - 4 bit cuda - 4 bit triton - 4 bit cpu https://github.com/oobabooga/text-generation-webui/blob/main... Models should be quantized specifically for each, and both branches are under…

comment by brucethemoose2 · 2023-04-23 · in Ask HN: Are there any ready-to-use Docker images for running LLMs locally?

This plus the the vicuna models is probably as close as you can get in the near term. https://github.com/oobabooga/text-generation-webui

comment by ospray · 2023-04-18 · in GPT4All Chat – Locally-running AI chat application powered by the GPT4All-J

…as well, but maybe I'll see if the extra cycles helps). I'm giving 30B a try on my GPU now with https://github.com/oobabooga/text-generation-webui/wiki/LLaM... and if it's not good then will give layer offloading with 65B a try and see if…

comment by lhl · 2023-04-08 · in State-of-the-Art Chatbot, Vicuna-7B, now runs on MacBook with GPU acceleration

The default loader doesn't seem to let you load quantized models but if you use something like https://github.com/oobabooga/text-generation-webui you can 1) use the model with `--load-in-8bit` which halves the memory (runs on my 24GB…

comment by lhl · 2023-04-05 · in State-of-the-art open-source chatbot, Vicuna-13B, just released model weights

…python bindings for llama.cpp and the text-generation-webui already uses them for local inference. https://github.com/oobabooga/text-generation-webui/wiki/llam... "pip install llamacpp" or https://github.com/thomasantony/llamacpp-python

comment by MacsHeadroom · 2023-04-04 · in An LLM playground you can run on your laptop

As an alternative for purely local LLMs, I've been having fun with this setup: https://github.com/oobabooga/text-generation-webui

comment by a5huynh · 2023-04-04 · in An LLM playground you can run on your laptop

https://github.com/oobabooga/text-generation-webui

comment by enlyth · 2023-03-31 · in Llama.cpp 30B runs with only 6GB of RAM now

…the discussion of that second paper here is interesting, though it doesn't have its own benchmarks: https://github.com/oobabooga/text-generation-webui/issues/17...

comment by corvec · 2023-03-31 · in Llama.cpp 30B runs with only 6GB of RAM now

…the same result. If you want to run larger Alpaca models on a low VRAM GPU, try FlexGen. I think https://github.com/oobabooga/text-generation-webui/ is one of the easier ways to get that going.

comment by sp332 · 2023-03-31 · in Llama.cpp 30B runs with only 6GB of RAM now

text-generation-webui ( https://github.com/oobabooga/text-generation-webui )

comment by nickthegreek · 2023-03-29 · in Gpt4all: A chatbot trained on ~800k GPT-3.5-Turbo Generations based on LLaMa

I was mostly referring to this project, which has some 1-click installers: https://github.com/oobabooga/text-generation-webui#alternati... Though I have not tried those 1-click installers, instead I have been manually running it. That…

comment by 2bitencryption · 2023-03-28 · in Cerebras-GPT: A Family of Open, Compute-Efficient, Large Language Models

OK, looks like the "LLM retard guide" is "run this installer": https://github.com/oobabooga/text-generation-webui/releases/... That's the only thing that worked for me, but it was so easy and worked instantly. It's a set of…

comment by LoganDark · 2023-03-27 · in FlexGen: Running large language models on a single GPU

Best way to mess around with FlexGen and LLMs on local hardware in general is https://github.com/oobabooga/text-generation-webui

comment by lxe · 2023-03-26 · in FlexGen: Running large language models on a single GPU