Hacker GitHub

← All repos

peak/s5cmd

View on GitHub ↗ ★ 4,228 Go

Parallel S3 and local filesystem execution tool.

9 mentions on Hacker News by 9 people: 1 story, 8 comments · first 2020-09-13 · latest 2025-06-11

Mentions per year 9 mentions · busiest year 2021 (3)
20082026
AllStoriesComments NewestTopOldest
  1. ▲ 62
    S5cmd: Parallel S3 and local filesystem execution tool
    story by polyrand · 2025-06-11 · 23 comments
  2. …link (~4.3 GB/s), whereas s3cmd and aws-cli can only reach 85 MB/s and 375 MB/s respectively. [1] https://github.com/peak/s5cmd
    comment by nodesocket · 2025-06-11 · in Show HN: S3mini – Tiny and fast S3-compatible client, no-deps, edge-ready
  3. …link (~4.3 GB/s), whereas s3cmd and aws-cli can only reach 85 MB/s and 375 MB/s respectively. [1] https://github.com/peak/s5cmd
    comment by iknownothow · 2024-07-09 · in Show HN: S3HyperSync – Faster S3 sync tool – iterating with up to 100k files/s
  4. …tank/pgdata@snapshot | pbzip2 > mybackup.zfs.bz2 s5cmd cp mybackup.zfs.bz2 s3://mygooglebucket/ https://github.com/peak/s5cmd/blob/master/README.md#Benchmar...
    comment by broner · 2024-06-07 · in We migrated from AWS to GCP with minimal downtime
  5. I use s5cmd, it’s a fast parallel s3 cli client: https://github.com/peak/s5cmd Haven’t heard of rclone before so can’t compare yet but s5cmd is supposed to work against GCS and other s3 compatible apis.
    comment by faizshah · 2022-10-07 · in Gcloud storage: Faster data transfers for Cloud Storage
  6. …in parallel goroutines. 0. https://docs.aws.amazon.com/AmazonS3/latest/userguide/batch-... 1. https://github.com/peak/s5cmd#delete-multiple-s3-objects
    comment by vdm · 2021-10-19 · in Deleting an S3 Bucket Costs Money
  7. …Go, performs cp in parallel to/from s3 too. I've been using it for some time and it's plenty fast. https://github.com/peak/s5cmd
    comment by hiyer · 2021-04-11 · in Downloading files from S3 with multithreading and Boto3
  8. …been using s5cmd which we’ve found is ridiculously faster than boto without any extra boto tricks. https://github.com/peak/s5cmd
    comment by catillac · 2021-04-10 · in Downloading files from S3 with multithreading and Boto3
  9. or even faster https://github.com/peak/s5cmd
    comment by gfody · 2020-09-13 · in Moving 25TB data from one S3 bucket to another took 7 engineers and 2 full days