hpc/dcp

View on GitHub ↗ ★ 198 Shell

dcp is a distributed file copy program that automatically distributes and dynamically balances work equally across nodes in a large distributed system without centralized state.

3 mentions on Hacker News by 1 person: 1 story, 2 comments · first 2012-03-09 · latest 2015-01-29

Mentions per year 3 mentions · busiest year 2012 (1)

20082026

All Stories Comments Newest Top Oldest

…interested in seeing my implementation (it's buggy, but handles most cases), it can be found at: https://github.com/hpc/dcp/blob/master/src/handle_args.c The entry point to handle_args.c is DCOPY_parse_path_args, which is called by the…
comment by fintler · 2015-01-29 · in Parsing command line arguments using a finite state machine and backtracking
…copy program at my last job to copy files in a reasonable time frame on 5PB to 55PB filesystems. https://github.com/hpc/dcp We got an IEEE paper out of it: http://conferences.computer.org/sc/2012/papers/1000a015.pdf A few people are…
comment by fintler · 2014-09-12 · in My experience with using cp to copy 432 million files (39 TB)