Hacker GitHub

← All repos

hpc/libcircle

View on GitHub ↗ ★ 101 C

An API to provide an efficient distributed queue on a cluster. Libcircle is currently used in production to quickly traverse and perform operations on a file tree which contains several hundred-million file nodes.

5 mentions on Hacker News by 2 people: 2 stories, 3 comments · first 2011-09-10 · latest 2026-07-26

Mentions per year 5 mentions · busiest year 2014 (2)
20082026
AllStoriesComments NewestTopOldest
  1. Do you know if fclones implements anything similar to https://github.com/hpc/libcircle ? I also wonder if nftw(3) would be faster than readdir(3).
    comment by ranger_danger · 2026-07-26 · in DskDitto: Ultra-fast, parallel duplicate-file detector
  2. …be available at http://fileutils.io/ relatively soon. We also had another tool written on top of https://github.com/hpc/libcircle that would gather metadata on a few hundred-million files in a few hours (we had to limit the speed so it…
    comment by fintler · 2014-09-12 · in My experience with using cp to copy 432 million files (39 TB)
  3. …for single-mount petascale filesystem treewalking that extends Dijkstra's original design: https://github.com/hpc/libcircle We have a detailed paper on it coming out at SC12.
    comment by fintler · 2012-09-25 · in Google Spanner's Most Surprising Revelation: NoSQL is Out and NewSQL is In