facebookresearch/faiss on Hacker News

why not use this? https://github.com/facebookresearch/faiss

comment by itake · 2025-09-13 · in Vector database that can index 1B vectors in 48M

how different is this compared to Facebook's open-source tool Faiss[1]? [1] https://github.com/facebookresearch/faiss/

comment by boywitharupee · 2024-07-02 · in GraphRAG is now on GitHub

…index on your data, and using Product Quantization (like PQ768x1 for binary features in this case) https://github.com/facebookresearch/faiss/wiki/The-index-fac...

comment by lsb · 2024-05-03 · in I’m writing a new vector search SQLite Extension

…difference between the types of binary index then it'd be great to explain what that is too. [0] https://github.com/facebookresearch/faiss/wiki/Binary-indexe...

comment by DougBTX · 2024-04-29 · in Haystack DB – 10x faster than FAISS with binary embeddings by default

…for local mode, which means embeddable https://github.com/qdrant/qdrant-client Or just use FAISS https://github.com/facebookresearch/faiss

comment by andre-z · 2024-04-05 · in Show HN: Chromem-go – Embeddable vector database for Go

…make sure to use flash attention if you have A100s or better and context length >= 2k Add FAISS ( https://github.com/facebookresearch/faiss ) if you need fast local RAG

comment by xianshou · 2024-04-04 · in Ask HN: Most efficient way to fine-tune an LLM in 2024?

https://github.com/facebookresearch/faiss/wiki/Vector-codecs has some good things available too.

comment by osmarks · 2024-01-25 · in New embedding models and API updates

…project. Would be nice to see how it compares to the built-in operations provided in Faiss. https://github.com/facebookresearch/faiss/wiki/How-to-make-F...

comment by dmezzetti · 2024-01-24 · in SimSIMD: Hardware-accelerated similarity metrics and distance functions

…txtai ( https://github.com/neuml/txtai ) with a Faiss backend. This Faiss wiki article might help ( https://github.com/facebookresearch/faiss/wiki/Indexing-1G-v... ). For example, a partial Faiss configuration with 4-bit PQ quantization…

comment by dmezzetti · 2023-11-26 · in Investing in new vector database development vs enhancing existing databases

…locally with something like faiss which has good multiplatform support and sufficient tutorials: https://github.com/facebookresearch/faiss Doing full retrieval-augmented generation (RAG) and getting LLMs to interpret the results has more…

comment by minimaxir · 2023-10-30 · in Approximate Nearest Neighbor Oh Yeah (Annoy)

…More Like This and semantic search are on the agenda, I'd expect it to take about an hour to get https://github.com/facebookresearch/faiss up and working, I could spend more time stuck on some "little" front end problem like getting…

comment by PaulHoule · 2023-09-17 · in Ask HN: Are there any unsolved problems with vector databases

…available in FAISS [2]. [1] https://ai.googleblog.com/2020/07/announcing-scann-efficient... [2] https://github.com/facebookresearch/faiss/wiki/Fast-accumula...

comment by twelfthnight · 2023-07-31 · in USearch: Smaller and faster single-file vector search engine

…[4] https://medium.com/@nils_reimers/openai-gpt-3-text-embedding... [5] https://github.com/facebookresearch/faiss

comment by vikp · 2023-07-12 · in Serverless Semantic Search, Free tier only

…people shove them into faiss, which handles most of the rest very well for small/medium datasets: https://github.com/facebookresearch/faiss

comment by mattnewton · 2023-07-12 · in Introduction to vector similarity search (2022)

…nature of the hardware, but the GPU version is more geared towards batch lookup in any case. https://github.com/facebookresearch/faiss/wiki/Fast-accumula... https://github.com/facebookresearch/faiss/wiki/Indexing-1M-v... We have not found…

comment by jhj · 2023-05-06 · in Billion-Scale Approximate Nearest Neighbor Search [pdf]

…vectors, you can look at methods like quantization and compression, both detailed here for Faiss - https://github.com/facebookresearch/faiss/wiki/Indexing-1G-v... Elasticsearch uses HNSW, not sure what options they have but…

comment by dmezzetti · 2023-05-05 · in What is a Vector Database? (2021)

Faiss has long discussed strategies for scaling to 1B - 1T records here - https://github.com/facebookresearch/faiss/wiki/Indexing-1G-v... There are plenty of options available to run your own local vector database, txtai is one of them.…

comment by dmezzetti · 2023-05-05 · in What is a Vector Database? (2021)

…decades of experience in relational database systems and reduce risk in production deployment. [1] https://github.com/facebookresearch/faiss

comment by jarulraj · 2023-04-30 · in Show HN: EVA – AI-Relational Database System

…something run a bunch of fee-per-use API calls on its own, why double up on getting charged? [0] https://github.com/facebookresearch/faiss

comment by bugglebeetle · 2023-04-15 · in BabyAGI: An Autonomous and Self-Improving Agent

Thanks. I just googled 'faiss ivf disk' and a couple of clicks later found this page: https://github.com/facebookresearch/faiss/wiki/Indexes-that-...

comment by rahimnathwani · 2023-04-13 · in Do you need a vector database?

What’s wrong with Faiss? https://github.com/facebookresearch/faiss

comment by jessehu · 2023-04-13 · in Do you need a vector database?

…to faiss as an implementation of cuda gpu backed indices, although you can use RAFT directly. [0] https://github.com/facebookresearch/faiss/pull/2521 [1] https://github.com/rapidsai/raft

comment by mshachkov · 2023-03-30 · in Faiss: A library for efficient similarity search

…though, as it iterates over it, it creates embeddings and then stores them in the FAISS store ( https://github.com/facebookresearch/faiss ) which was created to handle a large amount of embeddings. For the actual queries, it filters it…

comment by mpaepper · 2023-03-22 · in Your website's content -> Q&A bot / chatbot

…to optimize for. See this paper https://arxiv.org/abs/1702.08734 And this library that it describes https://github.com/facebookresearch/faiss Which is an optimal use of your time as you can install it in a minute with anaconda if you use…

comment by PaulHoule · 2023-03-11 · in Ask HN: What’s the optimal approach for KNN (K- Nearest Neighbor)?

…the rows most similar to row #2) There are libraries for efficient vector similarity search like https://github.com/facebookresearch/faiss

comment by jstx1 · 2023-02-16 · in Ask HN: Disk-backed NumPy for data analysis?

…FAISS indexed that don't all fit in memory but I've not quite figured out how to do that yet: https://github.com/facebookresearch/faiss/issues/2675

comment by simonw · 2023-02-07 · in OpenAI-to-SQLite

…on your workload (properties of vectors, size of dataset). See [1] for some more background. [1] https://github.com/facebookresearch/faiss/wiki/Guidelines-to...

comment by woodson · 2023-02-07 · in Storing OpenAI embeddings in Postgres with pgvector

…and are not as robust because it doesn't encapsulate an understanding of the semantics of the text. [1] github.com/facebookresearch/faiss [2] github.com/google-research/google-research/tree/master/scann [3] www.pinecone.io

comment by saintarian · 2022-11-13 · in Similarity search and deduplication at scale

…better (in terms of query latency/throughput) with specialized implementations as in eg FAISS ( https://github.com/facebookresearch/faiss/wiki/Binary-hashin... ).

comment by ajtulloch · 2022-10-19 · in Simple, Fast, and Scalable Reverse Image Search

…No need to do pairwise comparisons. Annoy claims to handle even up to 1,000 dimensions reasonably. https://github.com/facebookresearch/faiss/issues/95

comment by code_biologist · 2022-06-02 · in Entity Resolution: The most common data science challenge

…or a part of semantic search stack) is a trivial matter with open-source libraries like Faiss https://github.com/facebookresearch/faiss It takes 5 minutes to set up. You can search billion vectors on common hardware. For low-latency (up to…

comment by freediver · 2021-12-14 · in Find anything fast with Google's vector search technology

…o.ca/~rsalakhu/papers/semantic_final.p... Edit: also, talking about LSH, must check out FAISS library https://github.com/facebookresearch/faiss and the current SOTA http://ann-benchmarks.com/

comment by sayonaraman · 2021-09-28 · in Vectors are over, hashes are the future of AI

…Facebook's FAISS [1,2]. For everything else, sklearn is amazing. [1] https://faiss.ai/ [2] https://github.com/facebookresearch/faiss

comment by woko · 2021-09-14 · in Scikit-Learn Version 1.0

It's widely used in recommenders based on embeddings. See: https://github.com/spotify/annoy https://github.com/facebookresearch/faiss

comment by jalammar · 2021-06-24 · in Introduction to Locality-Sensitive Hashing

…vectors, also does the (distributed) searching. Here is a partial list of possible vector types: https://github.com/facebookresearch/faiss/wiki/Faiss-indexes

comment by BenoitP · 2021-04-22 · in Pgvector – vector similarity search for Postgres

…guess from the 'ivfflat' keyword that this project is serving vectors produced by FAISS[1] somehow? https://github.com/facebookresearch/faiss

comment by BenoitP · 2021-04-22 · in Pgvector – vector similarity search for Postgres

…a another note Facebook has a feature-rich, GPU-powered, scalable indexation engine called FAISS: https://github.com/facebookresearch/faiss

comment by BenoitP · 2021-04-09 · in Vald: A Highly Scalable Distributed Vector Search Engine

…[1] https://fasttext.cc/docs/en/language-identification.html [2] https://github.com/facebookresearch/faiss

comment by onurcel · 2020-10-23 · in First AI model that translates 100 languages without relying on English

…index sitting on top of sub-indices), and indeed is possibly required for the largest indices, e.g. https://github.com/facebookresearch/faiss/wiki/Indexing-1T-v...

comment by jhj · 2019-12-08 · in Indexing Billions of Text Vectors

…use of clustering, quantization, and centriod-difference tricks to make computation efficient. See https://github.com/facebookresearch/faiss/wiki/Faiss-indexes and related links for more information.

comment by blr246 · 2019-11-07 · in Using the linear distance operator in Postgres 12 to find the closest match

…eption.thorn.engineering/en/latest/examples/dedu... [2] https://authors.library.caltech.edu/7694/ [3] https://github.com/facebookresearch/faiss [4] https://github.com/spotify/annoy

comment by faustomorales · 2019-11-06 · in Show HN: Perceptual hashing tools for detecting child sexual abuse material

…from my understanding, they turn their vectors into some kind of hash that is then searched. [0] - https://github.com/facebookresearch/faiss [1] - https://github.com/yahoojapan/NGT

comment by gravypod · 2019-10-07 · in Show HN: Imagededup – Finding duplicate images made easy

…of some of these in a project called faiss [0] which does a relatively good job of this. [0] - https://github.com/facebookresearch/faiss

comment by gravypod · 2019-09-19 · in Challenges in Implementing a Full-Text-Search Engine

…set by some characteristics (e.g. gender, race). [1] https://cmusatyalab.github.io/openface/ [2] https://github.com/facebookresearch/faiss

comment by kilotaras · 2018-02-27 · in Facebook turned on face recognition silently

…[1] https://github.com/searchivarius/nmslib [2] https://github.com/spotify/annoy [3] https://github.com/facebookresearch/faiss [4] https://scaletext.com/

comment by Radim · 2018-02-11 · in Using k-d trees to efficiently calculate nearest neighbors in 3D vector space

…by libraries like the following. Nmslib: https://github.com/searchivarius/nmslib Faiss (Facebook): https://github.com/facebookresearch/faiss Annoy (Spotify): https://github.com/spotify/annoy

comment by jschmitz28 · 2018-01-18 · in Building Cross-Lingual End-to-End Product Search with Tensorflow

https://github.com/facebookresearch/faiss/blob/master/PATENT...

comment by aub3bhat · 2017-09-23 · in Relicensing React, Jest, Flow, and Immutable.js