LostRuins/koboldcpp on Hacker News

Rolling build of Koboldcpp now supports Krea2: * https://github.com/LostRuins/koboldcpp/releases/tag/rolling Model weights: * https://github.com/leejet/stable-diffusion.cpp/blob/master/d... In the launcher GUI, select 'Image Gen' and set…

comment by Eisenstein · 2026-06-25 · in Krea 2: SOTA open-weights 12B image model

…supports image generation, but you will have to wait for the next release for Krea2 support. * https://github.com/LostRuins/koboldcpp

comment by Eisenstein · 2026-06-25 · in Krea 2: SOTA open-weights 12B image model

…a kcppt file and open that and it will download everything you need and load it for you. Engine: * https://github.com/LostRuins/koboldcpp/releases/latest/ Kcppt files: * https://huggingface.co/koboldcpp/kcppt/tree/main

comment by Eisenstein · 2026-02-10 · in Qwen-Image-2.0: Professional infographics, exquisite photorealism

Try koboldcpp with the kcppt config file. The easiest way by far. Download the release here * https://github.com/LostRuins/koboldcpp/releases/tag/v1.103 Download the config file here * https://huggingface.co/koboldcpp/kcppt/resolve/main/z-i…

comment by Eisenstein · 2025-12-07 · in Z-Image: Powerful and highly efficient image generation model with 6B parameters

…bypassing the GUI; I've successfully run it as a system service on Windows using nssm. https://github.com/LostRuins/koboldcpp/releases Though there are a lot of roleplay-centric gimmicks in its feature set, its context-shifting feature is…

comment by rhet0rica · 2025-06-25 · in MCP in LM Studio

…/github.com/ggerganov/llama.cpp/blob/master/docs/dock... has a docker image but no examples to run it https://github.com/LostRuins/koboldcpp?tab=readme-ov-file#do... docker image was broken for me on 7800xt running rhel9…

comment by qamononep · 2024-10-02 · in AMD GPU Inference

…Model weights: MiniCPM-V-2_6-Q6_K_L.gguf, mmproj-MiniCPM-V-2_6-f16.gguf Inference: * https://github.com/LostRuins/koboldcpp/releases/tag/v1.75.2

comment by Eisenstein · 2024-09-26 · in Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

…' Set anything you want as an API key. Now you have a truly local version. * https://github.com/LostRuins/koboldcpp/releases * https://huggingface.co/bartowski/Meta-Llama-3.1-8B-Instruct-... * https://github.com/LostRuins/koboldcpp/blo

comment by Eisenstein · 2024-09-18 · in Scramble: Open-Source Alternative to Grammarly

…for distributed compute. 1 - https://www.deepbrainchain.org/ 2 - https://horde.koboldai.net/ 3 - https://github.com/LostRuins/koboldcpp

comment by InitEnabler · 2024-04-24 · in Any Online Communities on Local/Home AI?

…integrates Clip with a llama model. Koboldcpp can now do this with many models out of the box: * https://github.com/LostRuins/koboldcpp/releases/tag/v1.61.2

comment by Eisenstein · 2024-04-02 · in LLaMA now goes faster on CPUs

…- https://github.com/ollama-webui/ollama-webui - https://github.com/Chainlit/chainlit - https://github.com/LostRuins/koboldcpp - https://github.com/ParisNeo/lollms-webui/

comment by smnscu · 2024-02-28 · in Show HN: I made an app to use local AI as daily driver

…- https://pinokio.computer - https://www.nvidia.com/en-us/ai-on-rtx/chat-with-rtx-generat... - https://github.com/LostRuins/koboldcpp Misc: - https://faraday.dev (AI Characters): No UI / Command line (not for me): - https://ollama.com -…

comment by pentagrama · 2024-02-28 · in Show HN: I made an app to use local AI as daily driver

…react to your input. There's also SillyTavern that I have yet to try out. - https://faraday.dev/ - https://github.com/LostRuins/koboldcpp - https://github.com/SillyTavern/SillyTavern

comment by nigma · 2023-11-22 · in LM Studio – Discover, download, and run local LLMs

…hardware. There are projects like llama.cpp https://github.com/ggerganov/llama.cpp and koboldcpp: https://github.com/LostRuins/koboldcpp that can make running open source models pretty easy. As for where to find the models, you can look at…

comment by KronisLV · 2023-11-17 · in A failed AI girlfriend product, and my lessons

I got koboldcpp ( https://github.com/LostRuins/koboldcpp ) running locally on my M1 mac with 16gb. Use the guides on http://4chan.org/g/ in the AI Chatbot Thread. In the future, you will own and control access to your very own AI…

comment by narrator · 2023-11-16 · in A failed AI girlfriend product, and my lessons

…you aren't using it already, as well as some of the "dynamic" sampling like mirostat and dynatemp: https://github.com/LostRuins/koboldcpp/pull/464 I think these should work with nvidia's implementation if you just swap the sampling out…

comment by brucethemoose2 · 2023-10-31 · in Phind Model beats GPT-4 at coding, with GPT-3.5 speed and 16k context

…and Koboldcpp is the easiest and quite flexible: https://github.com/PygmalionAI/aphrodite-engine https://github.com/LostRuins/koboldcpp Picking a model to host is quite a rabbit hole, but I am very interested in SciPhi and Zephyr at the…

comment by brucethemoose2 · 2023-10-31 · in Mistral 7B,The complete Guide of the Best 7B model

Koboldcpp has an OpenAI (and kobold api) endpoint now, and supports grammar syntax like you said: https://github.com/LostRuins/koboldcpp The biggest catch is it doesn't support llama.cpp's continuous batching yet. Maybe soon?

comment by brucethemoose2 · 2023-10-11 · in Mistral 7B

…and literally one click, and KoboldAI, with the very fast and vram-efficient exllamav2 backend: https://github.com/LostRuins/koboldcpp https://github.com/henk717/KoboldAI - Hosts pick a quantized community LLM to run, which is (IMO) the…

comment by brucethemoose2 · 2023-10-09 · in Chat with, and help host, a free community LLM "horde"

Koboldcpp does this: https://github.com/LostRuins/koboldcpp/releases/tag/v1.44.2 They basically just ship executables for different llama.cpp backends and select the correct one with a python script, which is fine, as the executables are…

comment by brucethemoose2 · 2023-09-26 · in Ollama for Linux – Run LLMs on Linux with GPU Acceleration

Also https://github.com/LostRuins/koboldcpp The UI is relatively mature, as it predates llama. It includes upstream llama.cpp PRs, integrated AI horde support, lots of sampling tuning knobs, easy gpu/cpu offloading, and its basically…

comment by brucethemoose2 · 2023-08-28 · in ChatGPT Enterprise

…Some projects to look into are * llama.cpp - https://github.com/ggerganov/llama.cpp * KoboldCpp - https://github.com/LostRuins/koboldcpp * GPT4All - https://gpt4all.io/index.html llama.ccp will run LLMs that have been ported to the gguf…

comment by loudmax · 2023-08-28 · in Continue with LocalAI: An alternative to GitHub's Copilot that runs locally

…the model that you will see on TheBloke's download page for the model (very important). KoboldCPP: https://github.com/LostRuins/koboldcpp TheBloke: https://huggingface.co/TheBloke I would start with a 13b or 7b model quantized to 4-bits…

comment by Aerroon · 2023-08-17 · in Why host your own LLM?

…do it. * koboldcpp might be better, I just haven't tried it yet Hope that helps! Edit: just tried https://github.com/LostRuins/koboldcpp and it also works great. I should have started here probably. Compile with `make LLAMA_CLBLAST=1` run…

comment by throwing_away · 2023-08-16 · in How Is LLaMa.cpp Possible?

Path of least resistance: - Download koboldcpp: https://github.com/LostRuins/koboldcpp - Download your 70B ggml model of choice, for instance airoboros 70B Q3_K_L: https://huggingface.co/models?sort=modified&search=70b+ggml - Run Koboldcpp…

comment by brucethemoose2 · 2023-08-16 · in How Is LLaMa.cpp Possible?

…even a small laptop one, prompt ingestion is dramatically faster. Try the latest Kobold release: https://github.com/LostRuins/koboldcpp But to answer your question, the GGML CPU implementation is very good, and actually generating the…

comment by brucethemoose2 · 2023-08-07 · in Ask HN: How is Llama-2 optimized so much?

…a ggml file and then running it with koboldcpp (with opencl offloading) for maximum ease of use. https://github.com/LostRuins/koboldcpp You could get even better results using Japanese llama.cpp grammar, but I have not been down that…

comment by brucethemoose2 · 2023-07-28 · in Ask HN: Are there any LLMs specialised in translating Japanese to English?

…server also has good guides to help you get going, linked in the model card of most of his models. https://github.com/LostRuins/koboldcpp https://huggingface.co/TheBloke Edit: There's a bug with the newest Nvidia drivers that causes speed…

comment by knaik94 · 2023-06-18 · in OpenLLaMA 13B Released

…The pipeline is still relatively new though and is a bit involved. https://koboldai.org/discord https://github.com/LostRuins/koboldcpp

comment by knaik94 · 2023-06-18 · in OpenLLaMA 13B Released

There are many UIs for running locally, but the easiest is koboldcpp: https://github.com/LostRuins/koboldcpp Its a llama.cpp wrapper descended from the roleplaying community, but works fine (and performantly) for questioning and such. You…

comment by brucethemoose2 · 2023-06-18 · in OpenLLaMA 13B Released