Hacker GitHub

← All repos

meta/zstd

View on GitHub ↗ ★ 28,151 C

Zstandard - Fast real-time compression algorithm

165 mentions on Hacker News by 112 people: 28 stories, 137 comments · first 2016-08-31 · latest 2026-09-07

Mentions per year 165 mentions · busiest year 2018 (27)
20082026
AllStoriesComments NewestTopOldest
  1. …by the creator of Node.js) https://deno.land/ - zstd, a relatively new compression algorithm https://github.com/facebook/zstd https://quixdb.github.io/squash-benchmark/ - BLAKE2b, which is the new hashing function you should be using. It's…
    comment by shpx · 2020-01-05 · in Ask HN: What Technologies to Learn in 2020?
  2. …ratio and compression speed, but its decompression speed is significantly lower and closer to zlib. https://github.com/facebook/zstd#benchmarks AFAIK (I haven't looked much into it since 2018) it's not widely supported by CDNs, but at…
    comment by dictum · 2020-01-05 · in Now using Zstandard instead of xz for package compression
  3. I wish zstd supported seeking and partial decompression ( https://github.com/facebook/zstd/issues/395#issuecomment-535... ). We could then use it for hosting disk images as it would be a lot faster than xz which we currently use.
    comment by rwmj · 2020-01-05 · in Now using Zstandard instead of xz for package compression
  4. ▲ 2
    Zstandard v1.4.4
    story by rubinelli · 2019-11-06 · 0 comments
  5. …to what you're going to compress; I would guess they contain something resembling frequency tables. https://github.com/facebook/zstd has some description but doesn't explain precisely what the dictionary contains.
    comment by waterhouse · 2019-08-11 · in Hutter Prize: Compress a 100MB file to less than the current record of 16 MB
  6. …its use in production. And then there are two re-implementations in Java [1] and Go [2]. [0] https://github.com/facebook/zstd/tree/dev/doc/educational_de... [1] https://github.com/airlift/aircompressor/tree/master/src/mai... [2]…
    comment by felixhandte · 2019-07-20 · in Zstandard v1.4.1
  7. ▲ 60
    Zstandard v1.4.1
    story by felixhandte · 2019-07-19 · 15 comments
  8. …of input?). ---- It's also worth pointing out that zstd also has plenty of options to fiddle with: https://github.com/facebook/zstd/blob/dev/programs/zstd.1.md...
    comment by vthriller · 2019-04-17 · in Zstandard v1.4.0
  9. ▲ 158
    Zstandard v1.4.0
    story by terrelln · 2019-04-16 · 67 comments
  10. …by a guy at Facebook and seems to be owned by Facebook as well, since it's hosted on their github: https://github.com/facebook/zstd On the other hand, according to https://facebook.github.io/zstd it seems that zstd is part of the Linux…
    comment by Xolvix · 2019-03-25 · in Arch Linux propose changing compression method from xz to zstd
  11. …plots that I work with to evaluate zstd performance, e.g. [2]. [1] https://imgur.com/gU2Gdf6 [2] https://github.com/facebook/zstd/pull/1317#issuecomment-4260...
    comment by felixhandte · 2018-12-20 · in Improving compression at scale with Zstandard
  12. Ha, that just came up: https://github.com/facebook/zstd/issues/1087
    comment by felixhandte · 2018-12-19 · in Improving compression at scale with Zstandard
  13. …will be a benchmark you run yourself with your own data. And hopefully you like what you see! [1] https://github.com/facebook/zstd/releases/tag/v1.3.6
    comment by felixhandte · 2018-12-19 · in Improving compression at scale with Zstandard
  14. …* Zstd provides a wrapper around the zlib API so you could transparently switch to zstd. [1] [0] https://github.com/facebook/zstd/blob/dev/lib/zstd.h#L1409 [1] https://github.com/facebook/zstd/tree/dev/zlibWrapper
    comment by terrelln · 2018-12-19 · in Improving compression at scale with Zstandard
  15. There is a zlib wrapper included in the project : https://github.com/facebook/zstd/tree/master/zlibWrapper
    comment by Cyan4973 · 2018-12-19 · in Improving compression at scale with Zstandard
  16. This may help? https://github.com/facebook/zstd/issues/1155
    comment by aberoham · 2018-11-26 · in Zstandard – Fast real-time compression algorithm
  17. …--long=31 -T4 -10 It should be faster than lrzip + zstd and provide about the same results. [0] https://github.com/facebook/zstd/releases/tag/v1.3.2
    comment by terrelln · 2018-11-26 · in Zstandard – Fast real-time compression algorithm
  18. ▲ 228
    Zstandard – Fast real-time compression algorithm
    story by pierreneter · 2018-11-26 · 51 comments
  19. …should take exactly as much time as it takes to download the file via stream processing. Also: https://github.com/facebook/zstd ; better ratios, better speed.
    comment by dpc_pw · 2018-10-26 · in Parsing logs 230x faster with Rust
  20. The commit that dual-licensed under BSD and GPLv2 also "removed PATENTS clause": https://github.com/facebook/zstd/commit/4f73b3b55d83b082aad1...
    comment by electrum · 2018-10-03 · in Zstandard Compression and the application/zstd Media Type
  21. …git repositories had a patent "grant" document, there was some angst about what exactly this meant: https://github.com/facebook/zstd/issues/335 -- this was considered "resolved" when facebook decided to distribute the software under GPLv2,…
    comment by jepler · 2018-10-03 · in Zstandard Compression and the application/zstd Media Type
  22. …(unlike gzip) lets you set your window to be GBs if you want (look for "long-range match finder" in https://github.com/facebook/zstd/releases if you want more context); those window sizes would, of course, not be ideal on a low-end…
    comment by twotwotwo · 2018-10-03 · in Zstandard Compression and the application/zstd Media Type
  23. …or have very cheap compression using its fastest modes (hundreds of MB/s/core; see --fast modes at https://github.com/facebook/zstd/releases/tag/v1.3.4 ). brotli especially helped at the high end (static stuff you'll serve many times and…
    comment by twotwotwo · 2018-10-03 · in Zstandard Compression and the application/zstd Media Type
  24. ▲ 1
    Zstdandard v1.3.5 “Dictionary Edition”
    story by terrelln · 2018-06-28 · 0 comments
  25. How about zstd? https://github.com/facebook/zstd
    comment by d33 · 2018-06-26 · in ‘You can't use Brotli for dynamic content’ and other misconceptions
  26. ▲ 1
    Zstandard – Fast real-time compression algorithm
    story by thefox · 2018-03-31 · 0 comments
  27. It is mentioned at the bottom of the README, but it isn't very findable. I've opened a PR https://github.com/facebook/zstd/pull/1085 .
    comment by terrelln · 2018-03-30 · in Zstandard v1.3.4 – faster everything
  28. …was added for inclusion in the kernel, you may choose either GPLv2 or BSD. See the header file https://github.com/facebook/zstd/blob/dev/lib/zstd.h .
    comment by terrelln · 2018-03-30 · in Zstandard v1.3.4 – faster everything
  29. The library is dual licensed under plain BSD [1] and GPLv2 [2]. [1] https://github.com/facebook/zstd/blob/dev/LICENSE [2] https://github.com/facebook/zstd/blob/dev/COPYING
    comment by terrelln · 2018-03-30 · in Zstandard v1.3.4 – faster everything
  30. …that covers the entire corpus. In real world use cases the compression ratio can go from 3x to 9x: https://github.com/facebook/zstd#the-case-for-small-data-com... What was the finality on the issue of patents with this library? Is there an…
    comment by koolba · 2018-03-30 · in Zstandard v1.3.4 – faster everything
  31. …ratios. It's backed by a very fast entropy stage, provided by Huff0 and FSE library." 1: https://github.com/facebook/zstd
    comment by portmanteaufu · 2018-03-30 · in Zstandard v1.3.4 – faster everything
  32. ▲ 202
    Zstandard v1.3.4 – faster everything
    story by terrelln · 2018-03-30 · 33 comments
  33. ▲ 2
    Release Zstandard v1.3.4 – faster everything
    story by ot · 2018-03-29 · 0 comments
  34. …https://github.com/t3rmin4t0r/orc/tree/zstd [1] - https://issues.apache.org/jira/browse/ORC-46 [2] - https://github.com/facebook/zstd/issues/775
    comment by gopalv · 2018-01-25 · in Zstandard – Real-time data compression algorithm
  35. …yet. Does anyone know any better? It seems like we could use a better alternative to Snappy. [1] https://github.com/facebook/zstd/issues/395
    comment by lars_francke · 2018-01-25 · in Zstandard – Real-time data compression algorithm
  36. …the patents clause [1] and GPLv2 [2]. GPLv2 was added for inclusion in the Linux kernel. [1] https://github.com/facebook/zstd/blob/dev/LICENSE [2] https://github.com/facebook/zstd/blob/dev/COPYING
    comment by terrelln · 2018-01-25 · in Zstandard – Real-time data compression algorithm
  37. …with `zstd --long` for a 128 MB window, or `--long=windowLog` for a `2^windowLog` byte window. [1] https://github.com/facebook/zstd/releases/tag/v1.3.2
    comment by terrelln · 2018-01-25 · in Zstandard – Real-time data compression algorithm
  38. …$ time cat linux-4.14-rc6.tar.zst | zstd -d > out2 real 0m1.702s user 0m1.220s sys 0m0.520s [0]: https://github.com/facebook/zstd/blob/dev/doc/images/DCspeed...
    comment by Iburinoc · 2017-10-24 · in An ode to pack: gzip’s forgotten decompressor
  39. …the the normal zstd decompressor. There is a short article about it in the latest release notes https://github.com/facebook/zstd/releases/tag/v1.3.2
    comment by terrelln · 2017-10-23 · in An ode to pack: gzip’s forgotten decompressor
  40. ▲ 1
    Zstandard long range match finder
    story by terrelln · 2017-10-12 · 0 comments
  41. ▲ 2
    Zstandard long range match finder
    story by terrelln · 2017-10-10 · 0 comments
  42. They dual licensed under BSD and GPLv2: https://github.com/facebook/zstd/issues/335#issuecomment-323... I expect they did a thorough analysis to work out if any of it was either already patented, or patentable. That doesn't mean they don't…
    comment by Flenser · 2017-09-18 · in On React and WordPress
  43. Interestingly, Facebook did remove its PATENTS clause on another project they have recently - zstd - https://github.com/facebook/zstd/pull/801
    comment by 0x0 · 2017-09-15 · in On React and WordPress
  44. …length, match length, offset) triples. [1] https://github.com/Cyan4973/FiniteStateEntropy [2] https://github.com/facebook/zstd/blob/dev/doc/zstd_compressi...
    comment by terrelln · 2017-08-22 · in Understanding Asymmetric Numeral Systems
  45. ▲ 2
    Zstandard relicensed to BSD+GPLv2 dropped the PATENTS clause
    story by Twirrim · 2017-08-19 · 0 comments
  46. …opinion and leave your comment on the thread which discusses the license change over on GitHub? https://github.com/facebook/zstd/issues/775
    comment by markyuckerberg · 2017-08-19 · in Explaining React's license
  47. ▲ 2
    Zstandard removes PATENTS clause
    story by sanxiyn · 2017-08-19 · 1 comment
  48. I am very happy to report that ZSTD removed PATENTS clause: https://github.com/facebook/zstd/pull/801
    comment by sanxiyn · 2017-08-19 · in Explaining React's license
  49. …thing. There is an issue with a related discussion about it going for more than a year: https://github.com/facebook/zstd/issues/335 Last update is interesting. Someone did a search and looked for any patents FB filed for and couldn't find…
    comment by rdtsc · 2017-07-16 · in Apache Foundation disallows use of the Facebook “BSD+Patent” license
  50. …any Patent Assertion: (i) against Facebook or any of its subsidiaries or corporate affiliates... https://github.com/facebook/zstd/blob/dev/PATENTS
    comment by dmit · 2017-04-09 · in Performance of Solaris' ZFS LZ4 Compression