meta/rocksdb on Hacker News

…take a back seat. RocksDB for example does not fsync its WAL writes in the default configuration. https://github.com/facebook/rocksdb/wiki/WAL-Performance#non...

comment by Retr0id · 2026-06-18 · in My LSM tree was slower than a B-tree. Then I profiled it

…two bits more independent afaik. https://github.com/apache/parquet-format/blob/master/BloomFi... https://github.com/facebook/rocksdb/blob/main/util/bloom_imp... First one is useful for grasping the idea second one is more comprehensive.…

comment by ozgrakkurt · 2026-02-22 · in Two Bits Are Better Than One: making bloom filters 2x more accurate

Wanted to share what I think is a great example of this https://github.com/facebook/rocksdb/blob/main/util/ribbon_al...

comment by ozgrakkurt · 2025-08-12 · in Ask HN: What trick of the trade took you too long to learn?

…well explained in the comments and is the state of the art implementation as far as I know. https://github.com/facebook/rocksdb/blob/main/util/bloom_imp... https://github.com/facebook/rocksdb/blob/main/util/ribbon_al...

comment by ozgrakkurt · 2025-05-02 · in Bloom Filters

…[3]: The CRDT paper https://pages.lip6.fr/Marc.Shapiro/papers/RR-7687.pdf [4]: e.g. https://github.com/facebook/rocksdb/blob/main/db/log_reader.... [5]: the simultaneous post by Nikita Prokopov https://tonsky.me/blog/crdt-filesync/

comment by gritzko · 2024-06-25 · in Resilient Sync for Local First

…is likely more along the lines of LSM compaction, with expiry they likely use a moral equivalent of https://github.com/facebook/rocksdb/wiki/Compaction-Filter to actually do the deletion.

comment by yencabulator · 2024-05-31 · in Things you wish you didn't need to know about S3

…data structure. For example, you can see RocksDB using "fancy" structure to store its data [0]. [0] https://github.com/facebook/rocksdb

comment by vineyardmike · 2024-03-29 · in The Next Big Step in Mojo Open Source

…but seems potentially interesting at first glance. [1]: https://youtu.be/MsY-BafQgj4?t=442 [2]: https://github.com/facebook/rocksdb/wiki/MANIFEST

comment by magicalhippo · 2023-07-04 · in ZFS 2.2.0 (RC): Block Cloning merged

…have to be backwards compatible, a lot more alternatives with similar features are available, e.g. https://github.com/facebook/rocksdb run behind e.g. Node.js.

comment by Rochus · 2023-06-11 · in A Case of the MUMPS (2007)

…showing decent results, though buffered IO still wins in some cases from what I can tell. [1]: https://github.com/facebook/rocksdb/issues/9931 [2]: http://smalldatum.blogspot.com/2022/06/fixing-mmap-performan...

comment by magicalhippo · 2023-05-14 · in Using MMAP with RocksDB

…amplification then leveled compactions are sub-optimal. I would try the universal compaction. - https://github.com/facebook/rocksdb/wiki/Universal-Compactio...

comment by eclark · 2023-04-20 · in How RocksDB Works

…to bypass the I/O stack and optimize performance. RocksDB also has corresponding optimizations: https://github.com/facebook/rocksdb/wiki/PlainTable-Format

comment by l05k · 2023-04-20 · in How RocksDB Works

…bugs in every third-party library that we use. For example, there are bugs in RocksDB: https://github.com/facebook/rocksdb/issues?q=is%3Aissue+clic... Here are some bugs in JeMalloc: https://github.com/jemalloc/jemalloc/issues/2222…

comment by zX41ZdbW · 2023-02-02 · in Google's OSS-Fuzz expands fuzz-reward program

…your heart, do you think this story ends similarly if they decide to all do "modern C++" instead? https://github.com/facebook/rocksdb

comment by hu3 · 2022-08-24 · in Rust – A hard decision pays off

…atatracker.ietf.org/doc/html/rfc8878#section-5 ), but the bulk of the dictionary is unstructured. [1] https://github.com/facebook/rocksdb/wiki/Dictionary-Compress...

comment by felixhandte · 2022-06-01 · in The Smartest Compression Features on the Market

…weak guarantee by default too, writing to the write-ahead-log (WAL), but not performing fsyncs ( https://github.com/facebook/rocksdb/wiki/WAL-Performance ). They make a similar write amplification argument too (…

comment by ajhconway · 2022-05-28 · in SplinterDB: High performance embedded key-value store

…RocksDB Docs explain that they use a WAL+Manifest Log and I assume others do something similar: https://github.com/facebook/rocksdb/wiki/RocksDB-Overview#3-... Wish it was just mentioned at least off hand that clearly this isn't the whole…

comment by 0xCMP · 2022-02-09 · in Understanding LSM trees: What powers write-heavy databases (2020)

This appears to be FB’s implementation: https://github.com/facebook/rocksdb/blob/837705ad8011e249d5e... GPLv2 or Apache 2.0

comment by bradleyjg · 2021-07-11 · in Ribbon filter: Practically smaller than Bloom and Xor

…And yes, totally, I know at least RocksDB supports exactly that behavior [0]. [0] https://github.com/facebook/rocksdb/blob/12f11373554af219c51...

comment by felixhandte · 2021-01-27 · in LZ4 – Extremely fast compression

…by using new IO interfaces (io_uring), not by using threads. That approach seems right to me. See https://github.com/facebook/rocksdb/wiki/MultiGet-Performanc... . My understanding is that io_uring support is still a work in progress. We…

comment by petermattis · 2020-09-15 · in Pebble: A RocksDB Inspired Key-Value Store Written in Go

No. LSM/Rocksdb databases do "block compressions". Even better is dynamic dictionary compression: https://github.com/facebook/rocksdb/wiki/Dictionary-Compress...

comment by ddorian43 · 2020-04-03 · in PostgreSQL is the worlds’ best database

…facebook's benchmarks at their word here, ignoring how utterly synthetic these benchmarks read: https://github.com/facebook/rocksdb/wiki/Performance-Benchma... > much stronger resilience guarantees, You're mistaken. There is no resilience…

comment by geocar · 2020-03-12 · in Stages of denial in encountering K

https://github.com/facebook/rocksdb/pull/5881 I thought I saw a PR fly by for mysql but I can't find it now so maybe I was imagining things.

comment by fnord123 · 2020-03-11 · in Memory-Efficient Search Trees for Database Management Systems [pdf]

…of it... The RocksDB folks seem to be a lot more honest about the realities of tuning KV stores: https://github.com/facebook/rocksdb/wiki/RocksDB-Tuning-Guid... > Unfortunately, configuring RocksDB optimally is not trivial. Even we as…

comment by cdoxsey · 2019-01-26 · in Badger vs. RocksDB

…be merged into RocksDB at large. If the work is shared for others to use, it shouldn't be so scary. https://github.com/facebook/rocksdb/pull/2795/commits Personally, I think research papers like this should open source their work…

comment by Rafuino · 2018-06-06 · in Reducing DRAM Footprint with NVM in Facebook

Its a leveldb fork with substantial changes. For example, a non binary search file format option: https://github.com/facebook/rocksdb/wiki/PlainTable-Format Pull down the code some time. Everything and the kitchen sink is in there…

comment by cdoxsey · 2018-05-25 · in In Twitter’s early days, only one celebrity could tweet at a time

…(competitor) https://www.voltdb.com/blog/2017/10/02/comparing-fast-data-p... - RocksDB (competitor) https://github.com/facebook/rocksdb/wiki/performance-benchma... Is the the big boom in FoundationDB just that it is "now Open Source", or…

comment by marknadal · 2018-05-04 · in FoundationDB community highlights, two weeks in

…with users in our lounge area. We're working hard to support our open source community as well! https://github.com/facebook/rocksdb

comment by gfosco · 2018-03-05 · in Open-sourcing a 10x reduction in Apache Cassandra tail latency

…more libraries [3], so the likelihood of running into a licensing conflict seems a lot higher. [1] https://github.com/facebook/rocksdb/blob/master/db/db_impl.c... [2] https://meshedinsights.com/2017/07/16/apache-bans-facebooks-... [3]…

comment by antoncohen · 2017-12-19 · in Osquery: Change license to Apache 2.0 and GPLv2

…last, and when it compacts the table it'll apply the aggregations. Basically what you do with with https://github.com/facebook/rocksdb/wiki/merge-operator just with a few lines of SQL instead of a ton of custom code.

comment by justinsaccount · 2017-10-10 · in Redshift vs. BigQuery vs. Snowflake benchmark

They recently changed the RocksDB licence ( https://github.com/facebook/rocksdb/commit/3c327ac ) This reopenned the discussion about React licensing ( https://github.com/facebook/react/issues/10191 ) so maybe this will change soon.

comment by ttrunck · 2017-07-20 · in Things to learn in React before using Redux

…be > // found in the LICENSE file. See the AUTHORS file for names of contributors. CONTRIBUTING ( https://github.com/facebook/rocksdb/blob/master/CONTRIBUTING... ) presumably needs updating as well to cover the variations (including the…

comment by boulos · 2017-07-16 · in Facebook relicenses RocksDB under Apache 2 license

…removes the LICENSE file which had the original LevelDb (from Google) copyright and LICENSE notice: https://github.com/facebook/rocksdb/commit/3c327ac2d0fd50bbd... Did RocksDB finally remove every last piece of code from LevelDB?…

comment by boulos · 2017-07-16 · in Facebook relicenses RocksDB under Apache 2 license

RocksDB has fixed this https://github.com/facebook/rocksdb/commit/3c327ac2d0fd50bbd... now and moved to Apache / GPL dual license.

comment by chx · 2017-07-16 · in Apache Foundation disallows use of the Facebook “BSD+Patent” license

Facebook relicensed RocksDB to dual Apache and GPLv2 3 hours ago: https://github.com/facebook/rocksdb/pull/2589

comment by geofft · 2017-07-16 · in Apache Foundation disallows use of the Facebook “BSD+Patent” license

…to binary trees, btrees, etc. It's sort of magical how related all the storage structures are. [0] https://github.com/facebook/rocksdb/wiki/rocksdb-basics [1] https://blog.acolyer.org/2014/11/26/the-log-structured-merge... [2]…

comment by makmanalp · 2017-05-31 · in Damn Cool Algorithms: Log structured storage (2009)

…everywhere and the basis for facebook/rocksdb[2]. [1]: https://github.com/google/leveldb [2]: https://github.com/facebook/rocksdb

comment by irfansharif · 2017-03-13 · in Ask HN: What are some examples of good code?

Differences from leveldb: https://github.com/facebook/rocksdb/wiki/Features-Not-in-Lev...

comment by pfraze · 2016-12-19 · in RocksDB: A Persistent Key-Value Store for Flash and RAM Storage

…uses Facebook's RocksDB, which is also, I'm told, a pretty nice package. However, there is this: https://github.com/facebook/rocksdb/blob/master/PATENTS "The license granted hereunder will terminate, automatically and without notice, if…

comment by Diederich · 2016-06-23 · in ArangoDB 3.0 Release – A Solid Ground to Scale

…collaborate with the RocksDB team. Column families [0] seem like a great fit for time series. [0] https://github.com/facebook/rocksdb/wiki/Column-Families

comment by misframer · 2015-06-11 · in InfluxDB v0.9.0 released with developer and production support

…an in-memory dictionary will provide far fewer guarantees and features. [1] http://rocksdb.org/ [2] https://github.com/facebook/rocksdb/wiki/Third-party-languag...

comment by wicknicks · 2015-04-22 · in MongoDB and RocksDB at Parse

…[1] http://cockroachdb.org [2] https://docs.google.com/document/d/11k2EmhLGSbViBvi6_zFEiKzu... [3] https://github.com/facebook/rocksdb/wiki/Time-to-Live

comment by DocSavage · 2014-09-20 · in Ask HN: Key-Value Store Database with Historical Lookup?

…a file (i.e. segment). You can write your compaction code via compaction_filter_factory defined in https://github.com/facebook/rocksdb/blob/master/include/rock... 2. RocksDb also has a feature that allows an application to close a block…

comment by dhruba_b · 2013-11-16 · in RocksDB – A persistent key-value store for fast storage environments

The benchmark at https://github.com/facebook/rocksdb/wiki/Performance-Benchma... states that for LevelDB, "in 24 hours it inserted only 2 million key-values", and that "each key is of size 10 bytes, each value is of size 800 bytes". I…

comment by wbolster · 2013-11-15 · in RocksDB – A persistent key-value store for fast storage environments

That link should be: https://github.com/facebook/rocksdb/blob/master/db/db_impl.c... (not db_impl.c but .cc) I was wondering for a while if it was possible to do RAII in idiomatic c99 -- and it appears it isn't (or doesn't make as much…

comment by e12e · 2013-11-15 · in RocksDB – A persistent key-value store for fast storage environments

fixed your link: https://github.com/facebook/rocksdb/blob/master/db/db_impl.c... Take another look! There's a guard object used at the function scope to ensure the lock is released, and this block is bracketed to release and reacquire the…

comment by tsewlliw · 2013-11-15 · in RocksDB – A persistent key-value store for fast storage environments

I'm surprised that the C++ code is not using the RAII idiom in some obvious places. For example: https://github.com/facebook/rocksdb/blob/master/db/db_impl.c There are many places with bracketed calls to mutex_.Lock and mutex_.Unlock(). An…

comment by Patient0 · 2013-11-15 · in RocksDB – A persistent key-value store for fast storage environments

Very nice work, and the wiki is also quite nice -- I wish more projects had a page like https://github.com/facebook/rocksdb/wiki/Rocksdb-Architectur... . It's really nice to see a clear, terse summary of what makes this project interesting…

comment by snewman · 2013-11-15 · in RocksDB – A persistent key-value store for fast storage environments

https://github.com/facebook/rocksdb/issues/new is a 404. Appreciated when project is on github and open for issues/PR, etc.

comment by droid5 · 2013-11-15 · in RocksDB – A persistent key-value store for fast storage environments