openzfs/zfs on Hacker News

…many records), they are split into separate transactions and partial writes are then possible: https://github.com/openzfs/zfs/blob/34205715e1544d343f9a6414... Writes on ZFS cease to be atomic around approximately 32MB in size if I read the…

comment by ryao · 2025-01-23 · in Working with Files Is Hard (2019)

…packages available for a number of applications for arm64—for me, most notably, OpenZFS: https://github.com/openzfs/zfs/issues/14511

comment by geerlingguy · 2025-01-16 · in GitHub Linux ARM64 hosted runners now available for free in public repositories

I'm subscribed to this GitHub issue to follow the situation: https://github.com/openzfs/zfs/issues/12014 It might not be the only issue related to ZFS native encryption but, at the very least, as long as this one isn't closed I'm assuming…

comment by quectophoton · 2025-01-16 · in Nix – Death by a Thousand Cuts

…years ago, when its performance was truly horrendous). You can file bug reports against ZFS here: https://github.com/openzfs/zfs I suggest filing a bug report against APT. There is no reason for it to be doing fsync calls on every file it…

comment by ryao · 2025-01-15 · in ZFS 2.3 released with ZFS raidz expansion

…appears to be going wrong when non-raw sends of encrypted datasets are received by another system: https://github.com/openzfs/zfs/issues/12014 I do not believe anyone has figured out what is going wrong there. It has not been for lack of…

comment by ryao · 2025-01-15 · in ZFS 2.3 released with ZFS raidz expansion

…disks underneath ZFS. That is why ZFS began to purge the buffer cache on every flush 11 years ago: https://github.com/openzfs/zfs/commit/cecb7487fc8eea3508c3b6... That is how it still works today: https://github.com/openzfs/zfs/blob/fe44c5a…

comment by ryao · 2025-01-14 · in ZFS 2.3 released with ZFS raidz expansion

Except that swap on OpenZFS still deadlocks 7 years later ( https://github.com/openzfs/zfs/issues/7734 ) so you're still going to need LUKS for your swap anyway.

comment by Twey · 2025-01-14 · in ZFS 2.3 released with ZFS raidz expansion

True, but I have a gut feeling that a lot of these thorny issues would come up again: https://github.com/openzfs/zfs/issues/3582

comment by uniqueuid · 2025-01-14 · in ZFS 2.3 released with ZFS raidz expansion

I would be interested in a fine tune on OpenZFS: https://github.com/openzfs/zfs

comment by ryao · 2025-01-02 · in We fine-tuned Llama and got 4.2x Sonnet 3.5 accuracy for code generation

…b-trees would be faster if I switched the intra node operations to use Eytzinger ordered arrays: https://github.com/openzfs/zfs/commit/677c6f8457943fe5b56d7a... There are two ways to look at this Big O wise. One is that insertions and…

comment by ryao · 2025-01-01 · in Static search trees: faster than binary search

…for posting your original comment. The process of writing my reply gave me a flash of inspiration: https://github.com/openzfs/zfs/pull/16896 I doubt that this will make us as fast as ext4 at unlinking files in a single thread, but it…

comment by ryao · 2024-12-22 · in Why we use our own hardware

…for backup (huge win for mail datasets), would be cautious about using it in production: https://github.com/openzfs/zfs/issues/12014

comment by herf · 2024-12-22 · in Why we use our own hardware

…space conditions. The new figure is 96%. It is controlled by metaslab_df_free_pct in metaslab.c: https://github.com/openzfs/zfs/blob/zfs-2.2.0/module/zfs/met... Modification operations become slow when you are at/above 96% space filled,…

comment by ryao · 2024-12-22 · in Why we use our own hardware

…where they were going with with their initial statement until they define what they mean by stable. https://github.com/openzfs/zfs/releases/tag/zfs-2.2.6 Edit: changed sentence to make more sense Edit 2: And if we are to interpret stable…

comment by SirGiggles · 2024-12-12 · in QEMU with VirtIO GPU Vulkan Support

…what was released checks notes 1 hour ago. So I think GP was correct at the time of their post. https://github.com/openzfs/zfs/releases/tag/zfs-2.2.7

comment by hamandcheese · 2024-12-12 · in QEMU with VirtIO GPU Vulkan Support

…issue and has to be resolved by breaking the compiler’s optimization through a clever trick: https://github.com/openzfs/zfs/commit/d634d20d1be31dfa8cf06e... 12352 Similarly, GCC may delete a memcpy to a buffer about to be freed, although I…

comment by ryao · 2024-12-11 · in Making memcpy(NULL, NULL, 0) well-defined

…Some of our recent work includes major ZFS features such as Linux Containers support (OpenZFS 2.2: https://github.com/openzfs/zfs/pull/12263 ), and Fast Deduplication (OpenZFS 2.3: https://github.com/openzfs/zfs/discussions/15896 ). We're…

comment by klarainc · 2024-12-02 · in Ask HN: Who is hiring? (December 2024)

…free that a binary search can be. I devised this a couple years ago for use in ZFS’ btree code: https://github.com/openzfs/zfs/commit/677c6f8457943fe5b56d7a... Feel to have it under your choice of OSI approved license, including CC-0.

comment by ryao · 2024-11-21 · in Fastest Branchless Binary Search

I implemented a variation of this in ZFS’ btree implementation a few years ago: https://github.com/openzfs/zfs/commit/677c6f8457943fe5b56d7a... It only works if the comparator is inlined, which is why I had to use a macro and duplicate the…

comment by ryao · 2024-11-21 · in Fastest Branchless Binary Search

…or miss whether any particular one will be completely problem free or be a total nightmare. :( • https://github.com/openzfs/zfs/discussions/14793 Best to avoid them at all costs.

comment by justinclift · 2024-11-11 · in Western Digital SSDs and Legacy Macs

…something I'm missing. [1]: http://eworldproblems.mbaynton.com/posts/2014/zfs-block-poin... [2]: https://github.com/openzfs/zfs/issues/3582

comment by magicalhippo · 2024-10-30 · in OpenZFS deduplication is good now and you shouldn't use it

Also everyone ignores the publicly visible zfs repo/issues. Corruption https://github.com/openzfs/zfs/issues/16631 crash/corruption https://github.com/openzfs/zfs/issues/16626 crash https://github.com/openzfs/zfs/issues/16623 just from…

comment by viraptor · 2024-10-12 · in Switching customers from Linux to BSD because boring is good

…it around and 2) the size of the pool is limited by the size of the smallest disk in the pool. [1] https://github.com/openzfs/zfs/pull/15022

comment by mrighele · 2024-09-27 · in Why are my ZFS disks so noisy?

Not in a stable release yet, but check out https://github.com/openzfs/zfs/discussions/15896 if you have a need for that.

comment by rincebrain · 2024-09-14 · in My 71 TiB ZFS NAS After 10 Years and Zero Drive Failures

zvols on linux have known performance problems, see https://github.com/openzfs/zfs/issues/7631 -- most VM disk images on ZoL end up being regular files, for this reason. zfs send+receive of those such work just fine, but the bigger problem…

comment by yencabulator · 2024-08-11 · in Making Machines Move

Check it out: https://github.com/openzfs/zfs/releases

comment by Jach · 2024-07-27 · in Sqlitefs: SQLite as a Filesystem

Beware, ZFS often hangs on USB disconnections, forcing a reboot: https://github.com/openzfs/zfs/issues/3461

comment by yencabulator · 2024-07-26 · in Bcachefs, an Introduction/Exploration

…when enabling encryption because it was not really stable and people were running into crashes. https://github.com/openzfs/zfs/issues?q=is%3Aissue+label%3A%...

comment by frankjr · 2024-07-26 · in Bcachefs, an Introduction/Exploration

https://github.com/openzfs/zfs/tree/master/cmd/zpool/compati...

comment by nubinetwork · 2024-07-08 · in No more boot loader: Please use the kernel instead

…for new kernels: > Upcoming Linux 6.7 release will change a couple of interfaces. Adapt or die! * https://github.com/openzfs/zfs/pull/15681 And "dubious license"? Really? Given the incorporation of CDDL code of DTrace into macOS and…

comment by throw0101d · 2024-06-14 · in ChromeOS will soon be developed on large portions of the Android stack

…with DRAID where there are not 'dedicated' SPARE disks but 'spare block' spread across the disks: - https://github.com/openzfs/zfs/wiki/dRAID-HOWTO/001b44728e1a...

comment by vermaden · 2024-06-01 · in ZFS Resilver on SMR Drives

…means that all drives 'read' for the resilver and all drives 'write' for the resilver. More here: - https://github.com/openzfs/zfs/wiki/dRAID-HOWTO/001b44728e1a... ... also the Sequential Resilver helps even more: -…

comment by vermaden · 2024-06-01 · in ZFS Resilver on SMR Drives

Also, don't use WD NVMe drives with ZFS or btrfs: https://github.com/openzfs/zfs/discussions/14793 Significant numbers of problems reported, and WD support is pretty much telling people it's their own problem for not using Windows. (!)

comment by justinclift · 2024-05-17 · in WD Releases New 6TB 2.5-Inch External Hard Drives – First Upgrade in Seven Years

…of people who've reported problems with ZFS (and btrfs) with WD SN770 and a few other WD models: • https://github.com/openzfs/zfs/discussions/14793

comment by justinclift · 2024-05-09 · in Disaster Recovery with ZFS and Zrepl

…make sure not to snapshot bpool or it will brick your system and require a complete reinstall. https://github.com/openzfs/zfs/issues/13873

comment by nikisweeting · 2024-05-01 · in Ubuntu 24.04 LTS is so buggy you can't install the OS [video]

…them with ZFS or btrfs. Lots of reports of problems with WD NVMe drives on those file systems: https://github.com/openzfs/zfs/discussions/14793

comment by justinclift · 2024-05-01 · in WD SN740 2TB review: The M.2 2230 OEM SSD of choice

…WD Red SN700. Those WD drives seem to have some real issues, at least with ZFS and btrfs. :( https://github.com/openzfs/zfs/discussions/14793

comment by justinclift · 2024-04-05 · in Radxa's SATA HAT makes compact Pi 5 NAS

For more details about this slop space, see this comment: https://github.com/openzfs/zfs/blob/99741bde59d1d1df0963009b...

comment by MarkSweep · 2024-04-04 · in A disk so full, it couldn't be restored

…and breaking ZFS which then needs a while to catch up. For the latest reoccurrence of this, see https://github.com/openzfs/zfs/pull/15931 for Linux 6.8 compatibility. There's also the fact that not everything is OOB compatible with ZFS.…

comment by vladvasiliu · 2024-03-20 · in Bug hunting in Btrfs

IMO the "next fs" is just zfs. They somewhat recently merged RAIDZ expansion feature https://github.com/openzfs/zfs/pull/12225 and make regular improvements. If no file system has what you need today, zfs will probably be the first one to…

comment by komali2 · 2024-02-20 · in DiskClick: Ever wanted to hear old hard drive sounds

…don't miss any edge cases, at least on the functional side. But then you occasionally get bugs like https://github.com/openzfs/zfs/issues/15453 which can't easily be caught by test suites.

comment by nubinetwork · 2024-02-12 · in OpenZFS bug reports for native encryption

https://github.com/openzfs/zfs/issues/10769 I wasn't a fan of that change, it serves no purpose other than to sweep issues under the carpet (which can be said of any project that uses one). Which is funny because they have just as many…

comment by nubinetwork · 2024-02-12 · in OpenZFS bug reports for native encryption

…it needed to keep this closed-source in order to get ROI[2]. [1]: https://youtu.be/opW9KhjOQ3Q [2]: https://github.com/openzfs/zfs/issues/12119

comment by magicalhippo · 2024-02-03 · in SeaweedFS fast distributed storage system for blobs, objects, files and datalake

…from 6 years ago. Just a month after that there was a pretty big bug in ZFS that corrupted data. https://github.com/openzfs/zfs/issues/7401 Corresponding HN discussion at the time: https://news.ycombinator.com/item?id=16797644

comment by antongribok · 2024-01-30 · in Rclone syncs your files to cloud storage

https://github.com/openzfs/zfs/issues/15526#issuecomment-181... > zpool get all tank | grep bclone > kc3000 bcloneused 442M > kc3000 bclonesaved 1.42G > kc3000 bcloneratio 4.30x > My understanding is this: If the result is 0 for both…

comment by dist-epoch · 2023-12-26 · in A data corruption bug in OpenZFS?

…files being moved around where suddenly files are zero). [0] https://bugs.gentoo.org/917224 [1] https://github.com/openzfs/zfs/issues/15526 (referenced in the article)

comment by moviuro · 2023-12-26 · in A data corruption bug in OpenZFS?

…work, and I also couldn't convince myself it should. It seems to have worked out. [1] [1] - https://github.com/openzfs/zfs/commit/f375b23c026aec00cc9527...

comment by rincebrain · 2023-12-23 · in Ask HN: What's your "it's not stupid if it works" story?

https://github.com/openzfs/zfs/issues/7631 This is a long-standing issue with zvols which affects overall system stability, and has no real solution as of yet.

comment by Xorlev · 2023-12-17 · in ZFS Profiling on Arch Linux

More details on this discussion: https://github.com/openzfs/zfs/issues/6824#issuecomment-1817... . Basically he is too busy to continue developing the encryption features but is able to review the related works. There is also discussion…

comment by wb14123 · 2023-12-17 · in ZFS Profiling on Arch Linux

…that they are just having issues with certain kind of hard drives. There is a PR out for this: https://github.com/openzfs/zfs/pull/15588

comment by PaulCarrack · 2023-12-17 · in ZFS Profiling on Arch Linux