Hacker GitHub

← All repos

hartator/wayback-machine-downloader

View on GitHub ↗ ★ 5,965 Ruby

Download an entire website from the Wayback Machine.

22 mentions on Hacker News by 12 people: 5 stories, 17 comments · first 2015-08-15 · latest 2026-01-15

Mentions per year 22 mentions · busiest year 2016 (6)
20082026
  1. …with the organization as it exists. [1] It looks like this might exist at some level, e.g. https://github.com/hartator/wayback-machine-downloader , but I've been trying to use this for a couple of weeks and every day I try I get a HTTP 5xx…
    comment by philipkglass · 2026-01-15 · in Inside The Internet Archive's Infrastructure
  2. I used wayback-machine-downloader, I think you need one of the forks to make it work though. https://github.com/hartator/wayback-machine-downloader
  3. There is a fix for that. This might point the way: https://github.com/hartator/wayback-machine-downloader/issue...
  4. …on archive.org. I have used wayback-machine-downloader before for completely static sites before: https://github.com/hartator/wayback-machine-downloader/
  5. You can use wayback_machine_downloader to automate downloading the archived pages https://github.com/hartator/wayback-machine-downloader/
  6. Specifically: https://wiki.archiveteam.org/index.php?title=Restoring which mentions https://github.com/hartator/wayback-machine-downloader and also this tip: > This is undocumented, but if you retrieve a page with id_ after the datecode,…
  7. …it to archive.org, then later you can use wayback-machine-downloader to grab it from archive.org. https://github.com/hartator/wayback-machine-downloader
  8. …One nice way to do so ( handy for any site that you think may vanish off Way Back Machine ): https://github.com/hartator/wayback-machine-downloader
  9. There are a few out there ready to go. Here's one https://github.com/hartator/wayback-machine-downloader You just dump and sync to s3 and use terraform to provision the route53 and bucket setups. Yes they are mostly content sites. The…
  10. …org/web/20140607001646/https://www.quirk... You can use this tool to download the files for the site: https://github.com/hartator/wayback-machine-downloader/
  11. https://github.com/hartator/wayback-machine-downloader (I discuss it on my https://www.gwern.net/Search tutorial and use it every once in while eg to make my mirror of 'Climb Mount Improbable' https://www.gwern.net/docs/genetics/selection/w…
  12. They should be able to use https://github.com/hartator/wayback-machine-downloader and get at least a static version of the website back online.
    comment by 3dm0nd · 2017-06-09 · in Verelox Wiped by Ex-Admin
  13. …but to retrieve a backup for one of my dead websites. It was a fun project. Shameless plug: https://github.com/hartator/wayback-machine-downloader/
  14. wayback_machine_downloader www.c2.com -c 20 Ref: https://github.com/hartator/wayback-machine-downloader
    comment by hartator · 2016-10-14 · in The c2 wiki was down
  15. We don't have a logo yet! Download an entire website from the Internet Archive Wayback Machine. https://github.com/hartator/wayback-machine-downloader
  16. Ha fun, I've made a similar tool not so long ago: https://github.com/hartator/wayback-machine-downloader/
  17. I've been using this with great success too; https://github.com/hartator/wayback-machine-downloader