Hacker GitHub

← All repos

facebookresearch/dietgpu

View on GitHub ↗ ★ 397 Cuda archived

GPU implementation of a fast generalized ANS (asymmetric numeral system) entropy encoder and decoder, with extensions for lossless compression of numerical and other data types in HPC/ML applications.

4 mentions on Hacker News by 1 person: 1 story, 3 comments · first 2022-01-31 · latest 2025-04-25

Mentions per year 4 mentions · busiest year 2022 (2)
20082026
AllStoriesComments NewestTopOldest
  1. …), fpzip from LLNL ( https://computing.llnl.gov/projects/fpzip ) and my library dietgpu from 2021 ( https://github.com/facebookresearch/dietgpu ) which we used to speed training on a large GPU cluster by about 10% wall clock time overall…
    comment by jhj · 2025-04-25 · in Lossless LLM compression for efficient GPU inference via dynamic-length float
  2. …how the training works (it's lossless compression, it computes the same thing that it did before). https://github.com/facebookresearch/dietgpu
    comment by jhj · 2024-04-11 · in The simple beauty of XOR floating point compression
  3. …). I for one got up to 400 GB/s throughput on A100 GPUs in my implementation ( https://github.com/facebookresearch/dietgpu ), which to my knowledge is/was the fastest entropy coder out there on a commercially available hardware platform,
    comment by jhj · 2022-07-30 · in Parallelising Huffman decoding and x86 disassembly by synchronising prefix codes