mudler/LocalAI on Hacker News

…gives you a lot more control and support for additional formats (GGML, GGUF, GPTQ, ONNX, etc). https://github.com/mudler/LocalAI

comment by vunderba · 2025-04-11 · in Nvidia on NixOS WSL – Ollama up 24/7 on your gaming PC

braintrust-proxy: https://github.com/braintrustdata/braintrust-proxy LocalAI: https://github.com/mudler/LocalAI E.g promptfoo and chainforge have multi-LLM workflows. Promptfoo has a YAML configuration for prompts, providers,:…

comment by westurner · 2024-07-02 · in Show HN: Adding Mistral Codestral and GPT-4o to Jupyter Notebooks

…https://github.com/promptfoo/promptfoo ChainForge: https://github.com/ianarawjo/ChainForge LocalAI: https://github.com/go-skynet/LocalAI

comment by westurner · 2024-06-30 · in Chrome is adding `window.ai` – a Gemini Nano AI model right inside the browser

…ends: https://github.com/mckaywrigley/chatbot-ui https://github.com/oobabooga/text-generation-webui https://github.com/mudler/LocalAI And then connecting them to off-line models servers: - Ollama - llama.cpp And you should avoid closed…

comment by vunderba · 2024-02-28 · in Show HN: I made an app to use local AI as daily driver

https://github.com/mudler/LocalAI

comment by khimaros · 2024-01-20 · in Show HN: Voxos.ai – An Open-Source Desktop Voice Assistant

Please check out the following projects: https://github.com/jmorganca/ollama https://github.com/mudler/LocalAI

comment by sigoden · 2023-12-25 · in Ask HN: Set Up Local LLM

LocalAI can do that: https://github.com/mudler/LocalAI https://localai.io/features/openai-functions/

comment by iamjackg · 2023-12-06 · in Swap OpenAI with any open-source model

How does this compare to LocalAI? https://github.com/mudler/LocalAI

comment by westurner · 2023-11-20 · in AI Proxy to swap in any LLM while using OpenAI's SDK

…models on cheap Rockchip SOCs. Your CPU should be fine, even if it's a little outdated. LocalAI: https://github.com/mudler/LocalAI Some decent models to start with: TinyLlama (extremely small/fast): https://huggingface.co/TheBloke/TinyLlama…

comment by smoldesu · 2023-11-11 · in Local LLM's to run on old iMac / Hardware

Neither of this really requires OpenAI. You can do it with locally-running models via something like https://github.com/mudler/LocalAI

comment by eliben · 2023-11-11 · in Retrieval Augmented Generation in Go

…for the structured outputs. For example you can use LocalAI[0][1] to run models locally. [0] https://github.com/go-skynet/LocalAI [1] https://localai.io/features/openai-functions/

comment by jackmpcollins · 2023-09-27 · in Show HN: Magentic – Use LLMs as simple Python functions

There's also LocalAI[0] which allows the use of local LLMs with an OpenAI compatible API. [0] https://github.com/go-skynet/LocalAI

comment by michaelmior · 2023-09-26 · in Show HN: Magentic – Use LLMs as simple Python functions

…: https://github.com/ianarawjo/ChainForge LocalAI creates a GPT-compatible HTTP API for local LLMs: https://github.com/go-skynet/LocalAI Is it necessary to have an HTTP API for each model in a comparative study?

comment by westurner · 2023-09-09 · in Asking 60 LLMs a set of 20 questions

Enterprises can bring up https://github.com/go-skynet/LocalAI , run Llama or others and connect to them from their Promptly LLM apps - So spin up GPU instances and host whatever model in their VPC and it connects to your SaaS stack? What…

comment by mveertu · 2023-08-28 · in ChatGPT Enterprise

…data. We also support interacting with open source LLMs from the platform. Enterprises can bring up https://github.com/go-skynet/LocalAI , run Llama or others and connect to them from their Promptly LLM apps. We also provide support and…

comment by ajhai · 2023-08-28 · in ChatGPT Enterprise

…(a web ui w/ different backends) - https://github.com/ggerganov/llama.cpp (fast local LLM runner) - https://github.com/go-skynet/LocalAI (has an openai-compatible api)

comment by jmorgan · 2023-08-28 · in ChatGPT Enterprise

…localAI [1] and set your local environment "OPENAI_API_HOST" to localhost and it should work. [1] https://github.com/go-skynet/LocalAI

comment by syntaxing · 2023-08-26 · in Beating GPT-4 on HumanEval with a fine-tuned CodeLlama-34B

https://localai.io/ https://github.com/go-skynet/LocalAI

comment by compressedgas · 2023-08-23 · in Ask HN: What are you using to host local LLM models that are accessible via API?

…to be quite good at very specific things. https://github.com/oobabooga/text-generation-webui https://github.com/go-skynet/LocalAI https://github.com/gitmylo/audio-webui

comment by smcleod · 2023-08-14 · in ChatGPT costs 700k/day to run; may go bankrupt in 2024

…go-skynet/LocalAI[0] or lm-sys/FastChat[1] which can emulate an OpenAI API using local models. 0: https://github.com/go-skynet/LocalAI 1: https://github.com/lm-sys/FastChat/ Edit: idk if any of this support function calling tho

comment by pablo1107 · 2023-08-09 · in Show HN: Agentflow – Run Complex LLM Workflows from Simple JSON

…project that allows self-hosted can be easily called over the network. Open AI compatible interface https://github.com/go-skynet/LocalAI

comment by FloatArtifact · 2023-07-27 · in Show HN: Continue – Open-source coding autopilot

…API some of the models are already getting wrapped up in an open ai compatible rest interface. See https://github.com/go-skynet/LocalAI

comment by ianpurton · 2023-07-24 · in Ask HN: How do I train a custom LLM/ChatGPT on my own documents?

Some (not all) projects are indeed "copying" the OpenAI APIs; ex: https://github.com/go-skynet/LocalAI/issues/588

comment by ignoramous · 2023-07-10 · in GPT-4 API General Availability

And https://github.com/go-skynet/LocalAI https://github.com/juliooa/secondbrain https://github.com/louisgv/local.ai

comment by ignoramous · 2023-07-09 · in PdfGptIndexer: Indexing and searching PDF text data using GPT-2 and FAISS

…models or relying on nothing at all. My bot calls LocalAI, which uses the same API as OpenAI does: https://github.com/go-skynet/LocalAI Porting an app over to it's API only requires you to change the endpoint and the name of the model…

comment by smoldesu · 2023-06-07 · in Tell HN: I won't use Google's language models in production

…But it's what I'm using, and now I'm saving 10s of dollars on inferencing costs per month! [0] https://github.com/go-skynet/LocalAI I'm also using this to improve acceleration - https://cloudmarketplace.oracle.com/marketplace/en_US/adf.ta..…

comment by smoldesu · 2023-06-01 · in Falcon 40B LLM (which beats Llama) now Apache 2.0

…. And as others have mentioned, for providing an API that's compatible with OpenAI, https://github.com/go-skynet/LocalAI seems to be the frontrunner at the moment. --- For the project I'm working on (in bio) we're currently struggling with…

comment by deet · 2023-05-31 · in Ask HN: What's the best self hosted/local alternative to GPT-4?

…this question changes every week. For compatibility with the OpenAI API one project to consider is https://github.com/go-skynet/LocalAI None of the open models are close to GPT-4 yet, but some of the LLaMA derivatives feel similar to…

comment by simonw · 2023-05-31 · in Ask HN: What's the best self hosted/local alternative to GPT-4?

…can rot, for all I care. [0] https://en.wikipedia.org/wiki/Google_LLC_v._Oracle_America%2... . [1] https://github.com/go-skynet/LocalAI

comment by smoldesu · 2023-05-27 · in Tell HN: I'm grateful to OpenAI for opening AI to the public

…gpt4all.cpp, rwkv.cpp, whisper.cpp, vicuna, koala, gpt4all-j, cerebras and many others!]( https://github.com/go-skynet/LocalAI ) - [GitHub - paulpierre/RasaGPT: RasaGPT is the first headless LLM chatbot platform built on top of Rasa and…

comment by hodanli · 2023-05-22 · in PrivateGPT

Does it work with LocalAI [1] if you change the openai.api_base value to http://localhost:8080/ ? 1. https://github.com/go-skynet/LocalAI

comment by gdubya · 2023-05-19 · in CLI tools for working with ChatGPT and other LLMs

…airgap my inferencing machine if-needed, something OpenAI won't have until they offer on-prem. [0] https://github.com/go-skynet/LocalAI

comment by smoldesu · 2023-05-15 · in Ask HN: Why should I learn BERT/transformers/etc. when LLMs like GPT-4 exist?

…seamlessly with OpenAI API. Join the LocalAI community today and unleash your creativity! GitHub: https://github.com/go-skynet/LocalAI We are also on discord! Feel free to join our growing community! https://discord.gg/uJAeKSAGDy

comment by anton5mith2 · 2023-04-27 · in LocalAI: Local models on CPU with OpenAI compatible API

…seamlessly with OpenAI API. Join the LocalAI community today and unleash your creativity! GitHub: https://github.com/go-skynet/LocalAI We are also on discord! Feel free to join our growing community! https://discord.gg/uJAeKSAGDy

comment by mudler · 2023-04-24