Hacker GitHub

← All repos

internetarchive/brozzler

View on GitHub ↗ ★ 823 Python

brozzler - distributed browser-based web crawler

4 mentions on Hacker News by 4 people: 0 stories, 4 comments · first 2018-06-05 · latest 2019-10-19

Mentions per year 4 mentions · busiest year 2019 (3)
20082026
AllStoriesComments NewestTopOldest
  1. …what IA have had to say about any of this.) Update: from an IA response in this thread, "brozzler": https://github.com/internetarchive/brozzler
    comment by dredmorbius · 2019-10-19 · in The Wayback Machine: Fighting Digital Extinction in New Ways
  2. …most of our render-based crawling is done by a proxy-based recorder similar to what you theorize ( https://github.com/internetarchive/brozzler ).
    comment by dvanduzer · 2019-10-18 · in The Wayback Machine: Fighting Digital Extinction in New Ways
  3. …- https://github.com/oduwsdl/ipwb - https://getpolarized.io - https://github.com/internetarchive/brozzler And many more blog posts, articles, projects, and organizations on the wiki!
    comment by nikisweeting · 2019-04-04 · in Why there’s so little left of the early internet
  4. …Eg, headless browser crawling development and deployment to capture javascript-heavy sites ( https://github.com/internetarchive/brozzler); this is much more expensive than "classic" crawling. For more on increasing storage costs and the…
    comment by bnewbold · 2018-06-05 · in Archive.org and California to start a data sharing and preservation project