neuml/txtai on Hacker News

You can try txtai ( https://github.com/neuml/txtai ) with a Faiss backend. This Faiss wiki article might help ( https://github.com/facebookresearch/faiss/wiki/Indexing-1G-v... ). For example, a partial Faiss configuration with 4-bit PQ…

comment by dmezzetti · 2023-11-26 · in Investing in new vector database development vs enhancing existing databases

Adding txtai to the list https://github.com/neuml/txtai txtai is an all-in-one embeddings database for semantic search, LLM orchestration and language model workflows. txtai can satisfy most vector database use cases such as being a…

comment by dmezzetti · 2023-11-26 · in Investing in new vector database development vs enhancing existing databases

…for consideration. Couple relevant links. 1. https://neuml.hashnode.dev/custom-api-endpoints 2. https://github.com/neuml/txtai

comment by dmezzetti · 2023-11-03 · in What's the difference between LangChain, llama indexand others like autollm?

Project is open source and available on GitHub: https://github.com/neuml/txtai

comment by dmezzetti · 2023-11-03 · in txtai is an all-in-one embeddings database for semantic search and LLM workflows

If you want an easy way to evaluate Faiss, Hnswlib and Annoy vector backends, check out txtai - https://github.com/neuml/txtai . txtai also supports NumPy and PyTorch vector storage. Disclaimer: I am the author of txtai

comment by dmezzetti · 2023-10-30 · in Approximate Nearest Neighbor Oh Yeah (Annoy)

I'll add txtai to the list: https://github.com/neuml/txtai txtai is an all-in-one embeddings database for semantic search, LLM orchestration and language model workflows. Embeddings databases are a union of vector indexes (sparse and…

comment by dmezzetti · 2023-10-04 · in Choosing vector database: a side-by-side comparison

More info can be found below. GitHub: https://github.com/neuml/txtai Article: https://medium.com/neuml/introducing-txtai-the-all-in-one-em...

comment by dmezzetti · 2023-09-06 · in txtai: open-source embeddings database for semantic search and llm workflows

…some want to use their performance benchmarks for marketing. Disclaimer: I am the author of txtai ( https://github.com/neuml/txtai ), an open source embeddings database

comment by dmezzetti · 2023-09-05 · in Do we think about vector storage wrong?

…deciding what you need. As with anything complex, there often isn't a universal solution. txtai ( https://github.com/neuml/txtai ) can build indexes with Faiss, Hnswlib and Annoy. All 3 libraries have been around at least 4 years and are…

comment by dmezzetti · 2023-09-04 · in Vector Search with OpenAI Embeddings: Lucene Is All You Need

Another one mentioned below to try is txtai: https://github.com/neuml/txtai It can run embedded in a single Python instance and has no issues running in production that way.

comment by dmezzetti · 2023-08-20 · in Vector databases: analyzing the trade-offs

Adding txtai to the list: https://github.com/neuml/txtai txtai is an all-in-one embeddings database for semantic search, LLM orchestration and language model workflows. Embeddings databases are a union of vector indexes (sparse and dense),…

comment by dmezzetti · 2023-08-20 · in Vector databases: analyzing the trade-offs

…using Postgres and all you want is to add vector search, pgvector might be good enough. txtai ( https://github.com/neuml/txtai ) sets out to be an all-in-one embeddings database. This is more than just being a vector database with semantic…

comment by dmezzetti · 2023-08-12 · in Do we really need a specialized vector database?

…has a lot of important changes and is the base for a lot to come. See links below for more. GitHub: https://github.com/neuml/txtai Release Notes: https://github.com/neuml/txtai/releases/tag/v6.0.0 Article:…

comment by dmezzetti · 2023-08-11 · in Show HN: txtai: the all-in-one embeddings database - 6.0 released

The built-in JSON functionality is very powerful. txtai ( https://github.com/neuml/txtai ) takes full advantage of it and stores all relational data as JSON in SQLite. The ability to build indexes on these JSON functions is important.…

comment by dmezzetti · 2023-08-11 · in SQLite Functions for Working with JSON

…has a lot of important changes and is the base for a lot to come. See links below for more. GitHub: https://github.com/neuml/txtai Release Notes: https://github.com/neuml/txtai/releases/tag/v6.0.0 Article:…

comment by dmezzetti · 2023-08-10 · in txtai 6.0: the all-in-one embeddings database

txtai ( https://github.com/neuml/txtai ) is another option to consider. It has vector search with SQL, topic modeling and LLM prompt-driven search (retrieval augmented generation). Disclaimer: I am the author of txtai

comment by dmezzetti · 2023-07-31 · in Which vector database should I use? A comparison cheatsheet

If you'd like another LLM framework option that also is a vector database, check out txtai ( https://github.com/neuml/txtai ). Disclaimer: I am the author of txtai

comment by dmezzetti · 2023-07-14 · in The Problem with LangChain

…another option built on a foundation of open LLMs, open models and open source, check out txtai ( https://github.com/neuml/txtai ). Disclaimer: I am the author of txtai

comment by dmezzetti · 2023-07-08 · in Langchain Is Pointless

txtai makes it easy to use Hugging Face embeddings and Faiss, all local and configurable. https://github.com/neuml/txtai paperai is a sub-project focused on processing medical/scientific papers. https://github.com/neuml/paperai Disclaimer:…

comment by dmezzetti · 2023-07-08 · in PdfGptIndexer: Indexing and searching PDF text data using GPT-2 and FAISS

…We would love to hear the community's thoughts on the pros and cons of these two approaches. [1] https://github.com/neuml/txtai

comment by jarulraj · 2023-05-01 · in Show HN: EVA – AI-Relational Database System

…tuning a very small t5 model, notebook with usage samples (training code has to be somewhere near) https://github.com/neuml/txtai/blob/master/examples/33_Query...

comment by everlier · 2023-04-19 · in Ask HN: AI to study my DSL and then output it?

…SELECT id, text, date FROM txtai WHERE similar('machine learning') AND date >= '2023-03-30' GitHub: https://github.com/neuml/txtai This article is a deep dive on how the index format works: https://neuml.hashnode.dev/anatomy-of-a-txtai-inde…

comment by txtai · 2023-03-31 · in Faiss: A library for efficient similarity search

txtai combines SQLite and Faiss to enable vector search. It also does a lot more than that. https://github.com/neuml/txtai

comment by txtai · 2023-03-26 · in Vector database built for scalable similarity search

…- https://www.producthunt.com/posts/gitterbot-io-conversationa... - https://github.com/neuml/txtai/blob/master/examples/03_Build... - https://github.com/openai/openai-cookbook/blob/main/examples...

comment by thund · 2023-03-25 · in Open-Source GPT-4 Platform for Markdown

…ce.co/sentence-transformers/all-MiniLM-L6-v... is 384 dimensions and works great for semantic search. https://github.com/neuml/txtai https://neuml.github.io/txtai/embeddings/query/

comment by txtai · 2023-02-07 · in OpenAI-to-SQLite

GitHub repo: https://github.com/neuml/txtai

comment by txtai · 2023-02-03

…content. Read more: https://medium.com/neuml/neuspo-d42a6e33031 neuspo is powered by txtai ( https://github.com/neuml/txtai )

comment by neumll · 2022-11-05 · in Show HN: Summarize social media sports data with neuspo

There was a recent HN Post for codequestion which builds an offline semantic index (using https://github.com/neuml/txtai ) on the archive.org Stack Overflow dumps - https://news.ycombinator.com/item?id=33110219 GitHub:…

comment by txtai · 2022-10-20 · in The Overflow Offline project

…with improvements, including how to use BM25/TF-IDF indexes to drive topic models. Release notes: https://github.com/neuml/txtai/releases/tag/v5.1.0

comment by txtai · 2022-10-19 · in Txtai 5.1: better machine translation, Whisper transcription and ARM support

…with improvements, including how to use BM25/TF-IDF indexes to drive topic models. Release notes: https://github.com/neuml/txtai/releases/tag/v5.1.0

comment by txtai · 2022-10-18 · in Txtai 5.1 released: new translation models, Whisper transcription and ARM images

…Release Announcement - https://medium.com/neuml/whats-new-in-txtai-5-0-e5c75a13b101 Release Notes - https://github.com/neuml/txtai/releases/tag/v5.0.0

comment by txtai · 2022-10-01

…Release Announcement - https://medium.com/neuml/whats-new-in-txtai-5-0-e5c75a13b101 Release Notes - https://github.com/neuml/txtai/releases/tag/v5.0.0

comment by txtai · 2022-09-30

Source: https://github.com/neuml/txtai

comment by sts153 · 2022-09-29 · in Show HN: Txtai 5.0 released – build semantic search applications and workflows

…Release Announcement - https://medium.com/neuml/whats-new-in-txtai-5-0-e5c75a13b101 Release Notes - https://github.com/neuml/txtai/releases/tag/v5.0.0

comment by txtai · 2022-09-27 · in Txtai 5.0 released – build semantic search applications and workflows

https://github.com/neuml/txtai txtai can build vector indexes with Faiss/HNSW/Annoy and supports running SQL statements against them. External vector databases can also be plugged in.

comment by txtai · 2022-09-20 · in Measuring the popularity of different vector databases

GitHub Project Link: https://github.com/neuml/txtai

comment by txtai · 2022-07-06 · in Show HN: Build simple yet powerful open-source semantic applications

Everything in the notebook is open-source. It mainly uses the following projects: https://github.com/neuml/txtai https://github.com/kuprel/min-dalle txtai workflows can be containerized and run as a cloud serverless function -…

comment by txtai · 2022-07-03 · in Show HN: Generate webpage summary images with DALL-E mini

…article - https://medium.com/neuml/neuspo-d42a6e33031 Much of the logic in neuspo builds on txtai - https://github.com/neuml/txtai

comment by neumll · 2022-01-29 · in Machine learning to summarize social media sports data

….co/spaces/NeuML/txtai/tree/main/workflow... The above workflows can be built with this application - https://github.com/neuml/txtai/blob/master/examples/workflow... The application can create/save/load workflows and generates the workflow…

comment by neumll · 2022-01-20 · in No-code workflows for vector search

…good option, txtai has a notebook that shows how to export models for use in Rust/JavaScript/Java - https://github.com/neuml/txtai/blob/master/examples/18_Expor... Seems like a lot of tooling is being created in other languages besides…

comment by neumll · 2022-01-19 · in No-code workflows for vector search

Nice, good luck! All the source is Apache 2.0 - https://github.com/neuml/txtai

comment by neumll · 2022-01-19 · in No-code workflows for vector search

Link to GitHub project: https://github.com/neuml/txtai

comment by neumll · 2022-01-12 · in Vector Search with SQL

…( https://medium.com/neuml/neuspo-d42a6e33031 ) Much of the logic in neuspo builds on txtai ( https://github.com/neuml/txtai )

comment by neuml · 2020-11-29 · in Show HN: Neuspo – machine learning to discover fact-driven sports data