Hacker GitHub

← All repos

mhx/dwarfs

View on GitHub ↗ ★ 2,624 C++

A fast high-compression read-only file system for Linux, FreeBSD, macOS and Windows

19 mentions on Hacker News by 12 people: 3 stories, 16 comments · first 2020-11-29 · latest 2025-07-09

Mentions per year 19 mentions · busiest year 2024 (8)
20082026
AllStoriesComments NewestTopOldest
  1. …Links for your mentioned zpaq and dwarFS https://www.mattmahoney.net/dc/zpaq.html https://github.com/mhx/dwarfs
    comment by jauntywundrkind · 2025-07-09 · in Ptar: Replacing .tgz for petabyte-scale S3 archives
  2. …with a focus on achieving very high compression ratios in particular for very redundant data." https://github.com/mhx/dwarfs
    comment by tricked · 2024-12-20 · in Tell HN: Deduplicating a 10.4 TiB game preservation archive (WIP)
  3. I wonder if there is any benefit to using something like DwarFS ( https://github.com/mhx/dwarfs ) for something like this.
    comment by pskkk · 2024-06-13 · in You can help Anna's Archive by seeding torrents
  4. https://en.wikipedia.org/wiki/Locality-sensitive_hashing https://en.wikipedia.org/wiki/Nilsimsa_Hash https://github.com/mhx/dwarfs/blob/main/doc/mkdwarfs.md#nils...
    comment by mhx77 · 2024-04-12 · in DwarFS – Deduplicating Warp-Speed Advanced Read-Only File System
  5. https://github.com/mhx/dwarfs?tab=readme-ov-file#with-erofs It's been a while since I did that comparison, so the results could be significantly different now.
    comment by mhx77 · 2024-04-12 · in DwarFS – Deduplicating Warp-Speed Advanced Read-Only File System
  6. …page[1], you put the file at the filesystem creation: $ mkdwarfs -i /path/dir -o image.dwarfs [1]: https://github.com/mhx/dwarfs/blob/main/doc/mkdwarfs.md
    comment by claudex · 2024-04-12 · in DwarFS – Deduplicating Warp-Speed Advanced Read-Only File System
  7. Try this one? https://github.com/mhx/dwarfs It has a ton of comparison with existing tools in the README -- zpaqfranz included -- and it seems to be the best there is.
    comment by pdimitar · 2024-03-24 · in Pack: A new container format for compressed files
  8. I haven't had a chance to use it yet, but https://github.com/mhx/dwarfs claims to be times faster than squashfs, to compress much better, and to have full FUSE support.
    comment by pdimitar · 2024-03-23 · in Pack: A new container format for compressed files
  9. https://github.com/mhx/dwarfs "DwarFS compression is an order of magnitude better than SquashFS compression, it's 6 times faster to build the file system, it's typically faster to access files on DwarFS and it uses less CPU resources."
    comment by coldblues · 2022-07-24 · in Xz format considered inadequate for long-term archiving (2016)
  10. I thought DwarFS[0] is better than SquashFS? [0] https://github.com/mhx/dwarfs
    comment by pdimitar · 2022-07-24 · in Xz format considered inadequate for long-term archiving (2016)
  11. …[4]: https://github.com/jinfeihan57/p7zip [5]: https://github.com/mathieuchartier/mcm [6]: https://github.com/mhx/dwarfs/ [7]: https://github.com/Phantop/appdwarf/
    comment by phantop · 2022-02-25 · in Don't Use RAR
  12. Somewhat related (and definitely born out of a very similar use case): https://github.com/mhx/dwarfs I initially built this for having access to 1000+ Perl installations (spanning decades of Perl releases). The compression in this case is…
    comment by mhx77 · 2021-11-19 · in Elfshaker: Version control system fine-tuned for binaries
  13. Yes, and I've documented this now: https://github.com/mhx/dwarfs/blob/main/doc/dwarfs.md#settin...
    comment by mhx77 · 2020-11-30 · in DwarFS: A fast high compression read-only file system
  14. …in rather big blocks, some of the 16M blocks can actually be compressed down to less then 100k. [1] https://github.com/mhx/dwarfs/blob/main/thrift/metadata.thri...
    comment by mhx77 · 2020-11-30 · in DwarFS: A fast high compression read-only file system
  15. Read-only media, for example. Or in general, stuff that doesn't really change. In my case: https://github.com/mhx/dwarfs#history
    comment by mhx77 · 2020-11-29 · in DwarFS: A fast high compression read-only file system
  16. Whew! It was easy to find out how you actually initialize this thing, if it's read-only: https://github.com/mhx/dwarfs/blob/main/man/mkdwarfs.md
    comment by smitty1e · 2020-11-29 · in DwarFS: A fast high compression read-only file system