oobabooga/textgen on Hacker News

…from huggingface as gguf files and would like to use those downloads (and running them through https://github.com/oobabooga/text-generation-webui ). It is possible to run those llms as a local api using something like llama-cpp-python (…

comment by almaira · 2025-10-26 · in Is there an IDE that can use the local open-source model?

…text gen models locally that people might be interested in: https://www.reddit.com/r/LocalLlama [0] https://github.com/oobabooga/text-generation-webui [1] https://huggingface.co/Qwen/Qwen3-8B-GGUF

comment by Sophira · 2025-08-18 · in When you're asking AI chatbots for answers, they're data-mining you

…has a UI, but I haven't used it. I had some SSH tunnels set up to use the server interface with https://github.com/oobabooga/text-generation-webui where I hacked an "OpenAI" client to it (that UI doesn't have it natively). The only reason…

comment by adeon · 2025-01-28 · in Questions censored by DeepSeek

…~~decoder~~ (edit: sampler) level, like for any models? (see DRY ~~decoder~~ sampler for instance: https://github.com/oobabooga/text-generation-webui/pull/5677 )

comment by littlestymaar · 2025-01-28 · in Run DeepSeek R1 Dynamic 1.58-bit

Still nothing better than oobabooga ( https://github.com/oobabooga/text-generation-webui ) in terms of maximalism/"Pro"/"Prosumer" LLM UI/UX ALA Blender, Photoshop, Final Cut Pro, etc. Embarrassing and any VCs reading this can contact me…

comment by Der_Einzige · 2024-12-29 · in How I run LLMs locally

…and AUTOMATIC1111 work in the same way. They also have the same issues regarding Windows Defender. https://github.com/oobabooga/text-generation-webui https://github.com/AUTOMATIC1111/stable-diffusion-webui

comment by vulcanidic · 2024-11-29 · in Show HN: Voice-Pro – AI Voice Cloning

…implies local models and that you have a decent cpu + min 64gb of ram to run above 7b-sized model. https://github.com/oobabooga/text-generation-webui https://huggingface.co/models?pipeline_tag=text-generation&s...

comment by wruza · 2024-11-15 · in OpenAI, Google and Anthropic are struggling to build more advanced AI

…by the roleplaying/waifu sounding frontend names. They are all fine. This is what I mostly prefer: https://github.com/oobabooga/text-generation-webui

comment by dizhn · 2024-09-05 · in Yi-Coder: A Small but Mighty LLM for Code

…about the closures. Take a look at this issue to see what it takes to keep something open: https://github.com/oobabooga/text-generation-webui/issues/41... (not especially proud of my reactions there, but I hate being abused, even by…

comment by LoganDark · 2024-08-21 · in I've built my first successful side project, and I hate it

Why do people recommend this instead of the much better oobabooga text-gen-webui? https://github.com/oobabooga/text-generation-webui It's like you hate settings, features, and access to many backends!

comment by Der_Einzige · 2024-07-16 · in Codestral Mamba

I’m using a local frontend: https://github.com/oobabooga/text-generation-webui With local 8x7b: https://huggingface.co/TheBloke/Mixtral-8x7B-Instruct-v0.1-G... (Q4_K_M) Requires 35gb+ of regular RAM, RTX card is optional but speeds it up a…

comment by wruza · 2024-07-13 · in Ask HN: Fellow neurodivergents, do you pair better with LLMs than neurotypicals?

…use (fetching URLs and doing things with them) or RAG (searching your documents). I think Oobabooga https://github.com/oobabooga/text-generation-webui offers the latter through plugins. Our tool, https://github.com/transformerlab/transforme…

comment by aliasaria · 2024-04-01 · in Ask HN: What is the current (Apr. 2024) gold standard of running an LLM locally?

You can use webui https://github.com/oobabooga/text-generation-webui Once you get a version up and running I make a copy before I update it as several times updates have broken my working version and caused headaches. a decent explanation…

comment by instagib · 2024-03-17 · in Ask HN: How to get started with local language models?

…ordered by stars: - https://github.com/nomic-ai/gpt4all - https://github.com/imartinez/privateGPT - https://github.com/oobabooga/text-generation-webui - https://github.com/FlowiseAI/Flowise - https://github.com/lobehub/lobe-chat -…

comment by smnscu · 2024-02-28 · in Show HN: I made an app to use local AI as daily driver

…only be looking at open source chat UI front ends: https://github.com/mckaywrigley/chatbot-ui https://github.com/oobabooga/text-generation-webui https://github.com/mudler/LocalAI And then connecting them to off-line models servers: -…

comment by vunderba · 2024-02-28 · in Show HN: I made an app to use local AI as daily driver

…you want a truly general purpose front-end for LLMs, the only good solution right now is oobabooga: https://github.com/oobabooga/text-generation-webui All other alternatives have only small fractions of the features that oobabooga…

comment by Der_Einzige · 2024-02-27 · in Ask HN: People who switched from GPT to their own models. How was it?

…user who knows they have an Nvidia card" level. Assuming they even know how to find it: 1) Go to https://github.com/oobabooga/text-generation-webui?tab=readm... 2) See a bunch of instructions opening a terminal window and running random…

comment by kkielhofner · 2024-02-14 · in Nvidia's Chat with RTX is an AI chatbot that runs locally on your PC

The example waifu in text-generation-webui is good enough for me. https://github.com/oobabooga/text-generation-webui/blob/main...

comment by jjmarr · 2024-02-14 · in Your AI Girlfriend Is a Data-Harvesting Horror Show

…decisions to me. [0] https://huggingface.co/deepseek-ai/deepseek-coder-6.7b-instr... [1] https://github.com/oobabooga/text-generation-webui

comment by accrual · 2024-02-06 · in Ask HN: What are your top 3 coolest software engineering tools?

You can download it and run it with [this]( https://github.com/oobabooga/text-generation-webui ). There's an API mode that you could leverage from your VS Code extension.

comment by marinhero · 2024-01-29 · in Meta AI releases Code Llama 70B

…issues[1] so I am doing my own in Langroid using HuggingFace tokenizer.apply_chat_template [2] [1] https://github.com/oobabooga/text-generation-webui/issues/53... [2] https://github.com/langroid/langroid/blob/main/langroid/lang... Related…

comment by d4rkp4ttern · 2024-01-25 · in Ollama releases Python and JavaScript Libraries

https://github.com/oobabooga/text-generation-webui/blob/main... Consider a finetune - they're faster and relatively cheap (like, under $30 rented compute time). The link above lists them, but the steps are to gather a dataset, do the…

comment by washadjeffmad · 2024-01-02 · in Ask HN: Is it feasible to train my own LLM?

…1. Install llama.cpp from https://github.com/ggerganov/llama.cpp . Alternatively, install https://github.com/oobabooga/text-generation-webui 2. Go to https://huggingface.co/TheBloke and search for GGUF. Download and put the model file in…

comment by patrakov · 2023-12-29 · in AirLLM enables 8GB MacBook run 70B LLM

…either yet. EDIT: It's very slow, according to the comments in this thread by people who tried it: https://github.com/oobabooga/text-generation-webui/issues/47...

comment by rahimnathwani · 2023-12-28 · in AirLLM enables 8GB MacBook run 70B LLM

…much easier to get people to submit issues than to help solve them. [0] eg, windows install script https://github.com/oobabooga/text-generation-webui/blob/main...

comment by eyegor · 2023-12-26 · in Stirling-PDF: local web application to perform various operations on PDFs

All this needless difficulty when you can just do the right thing and use oogabooga. https://github.com/oobabooga/text-generation-webui

comment by Der_Einzige · 2023-12-18 · in Many options for running Mistral models in your terminal using LLM

https://github.com/oobabooga/text-generation-webui

comment by rahimnathwani · 2023-12-17 · in Run Mistral 7B on M1 Mac

…url with `OPENAI_API_BASE` and point it to a text-generation-ui OpenAI API[0] compliant model. 0: https://github.com/oobabooga/text-generation-webui/discussio...

comment by tetris11 · 2023-12-12 · in Role-playing with AI will be a powerful tool for writers and educators

https://github.com/oobabooga/text-generation-webui

comment by plasticchris · 2023-12-10 · in Ask HN: What's the best hardware to run small/medium models locally?

Already works on oobabooga as of a few days ago: https://github.com/oobabooga/text-generation-webui/issues/47... Need a few extra steps: https://github.com/oobabooga/text-generation-webui/pull/4803

comment by lxe · 2023-12-09 · in QuIP#: 2-bit Quantization for LLMs

Probably this because it's a simple UI to get you started: https://github.com/oobabooga/text-generation-webui

comment by politelemon · 2023-11-24 · in Generative AI for Beginners

…only. If you want keyboard shortcuts or a snappy interface, Python+gradio just isn't it. https://github.com/oobabooga/text-generation-webui/issues/41...

comment by LoganDark · 2023-11-23 · in LM Studio – Discover, download, and run local LLMs

Thats why as a business i would rather use a trusted FOSS LLm interface like textgeneration-WEBUI https://github.com/oobabooga/text-generation-webui

comment by busssard · 2023-11-22 · in LM Studio – Discover, download, and run local LLMs

…them appropriately for different LLMs. For instance, see TUI and its instruction template presets: https://github.com/oobabooga/text-generation-webui/tree/main...

comment by brucethemoose2 · 2023-11-01 · in MistralLite by Amazon Web Services

I use https://github.com/oobabooga/text-generation-webui separately as I code in VS 2022, VS Code, Intellij Idea and Webstorm

comment by SillyUsername · 2023-10-18 · in LLMs up to 4x Faster With Latest NVIDIA Drivers on Windows

…other runner for the model that supports grammars or the other features you are looking for.) [0] https://github.com/oobabooga/text-generation-webui

comment by dragonwriter · 2023-10-11 · in Mistral 7B

…do a one-off train for a model, you might be able to get away with doing it in something like this: https://github.com/oobabooga/text-generation-webui Very easy to use project but it really doesn't allow for the kind of metrics, analysis,…

comment by TrueDuality · 2023-09-28 · in Mistral 7B

…end users who run these LLMs on their desktops to roleplay. AMD being awful is orthogonal here. [1] https://github.com/oobabooga/text-generation-webui

comment by redox99 · 2023-09-28 · in Mistral 7B

OobaBooga supports this kind of load-and-go LORA: https://github.com/oobabooga/text-generation-webui

comment by mikeravkine · 2023-09-04 · in Show HN: finetune LLMs via the Finetuning Hub

I like this one: https://github.com/oobabooga/text-generation-webui LLM is the swiss army knife for everyone not just developers.

comment by prakhar897 · 2023-08-29 · in DevToys – A Swiss army knife for developers

You should be able to turn on the API in booba: https://github.com/oobabooga/text-generation-webui#api

comment by thelastparadise · 2023-08-28 · in Continue with LocalAI: An alternative to GitHub's Copilot that runs locally

…- I'm a maintainer) - https://github.com/simonw/llm (access LLMs from the cli - cloud and local) - https://github.com/oobabooga/text-generation-webui (a web ui w/ different backends) - https://github.com/ggerganov/llama.cpp (fast local LLM…

comment by jmorgan · 2023-08-28 · in ChatGPT Enterprise

Not hard with text-generation-ui! https://github.com/oobabooga/text-generation-webui I personally use llama.cpp as the driver since I run CPU-only but another may be better suited for GPU usage. But then it's as simple as downloading the…

comment by sodality2 · 2023-08-26 · in Beating GPT-4 on HumanEval with a fine-tuned CodeLlama-34B

Did you try Oobabooga ( https://github.com/oobabooga/text-generation-webui ) yet?

comment by MaKey · 2023-08-24 · in Code Llama, a state-of-the-art large language model for coding

…already I'm not sure. For general chat testing/usage this works pretty well with lots of options - https://github.com/oobabooga/text-generation-webui/

comment by thewataccount · 2023-08-24 · in Code Llama, a state-of-the-art large language model for coding

…version. You should be able to fit the original 70B with "load_in_8bit" on one A100 80GB. [1] https://github.com/oobabooga/text-generation-webui [2] https://www.runpod.io/ [3] https://gpus.llm-utils.org/running-llama-2-on-runpod-with-oo...

comment by syntaxing · 2023-08-23 · in GPT-3.5 Turbo fine-tuning and API updates

I've seen this mentioned in some guides I have read (I believe this is the one you are referencing: https://github.com/oobabooga/text-generation-webui ) and will definitely look into it, thanks!

comment by Ms-J · 2023-08-17 · in Ask HN: Local LLM's

…instructions: https://huggingface.co/Open-Orca/OpenOrcaxOpenChat-Preview2-... easiest way would be https://github.com/oobabooga/text-generation-webui a little more complex way I do is I have a stack with llama.cpp server, a openai adapter,…

comment by avereveard · 2023-08-16 · in Show HN: LlamaGPT – Self-hosted, offline, private AI chatbot, powered by Llama 2

…AI histira was a little more rampant but still. tl;dr my recommendation for your pdf project is run https://github.com/oobabooga/text-generation-webui . if your can get a 30 series GPU in your company Then run a 13B 4bit model that can…

comment by rig666 · 2023-08-16 · in Why host your own LLM?

…little messy and cobbled together I run oobabooga's API on docker with a 13B 4bit quantized model. https://github.com/oobabooga/text-generation-webui We use GTX 3060s because there the best bang for there buck in terms of VRAM. Our current…

comment by rig666 · 2023-08-15 · in Why host your own LLM?