yujiosaka/headless-chrome-crawler
View on GitHub ↗ ★ 5,631 JavaScript
Distributed crawler powered by Headless Chrome
5 mentions on Hacker News by 5 people: 1 story, 4 comments · first 2018-02-22 · latest 2020-10-26
20082026
-
…and it works pretty well. Not sure how well it will scale, but I'll index a few hundred sites only. https://github.com/yujiosaka/headless-chrome-crawler
comment by tiborsaas · 2020-10-26 · in Web Scraping with JavaScript -
Also check out Headless Chrome Crawler if you want to crawl a site. https://github.com/yujiosaka/headless-chrome-crawler
comment by jamieweb · 2018-10-20 · in Ask HN: What are you working on and why is it cool? -
Looks like this is actually built on top of puppeteer. See the "Note" under "Installation": https://github.com/yujiosaka/headless-chrome-crawler/blob/ma...
comment by itsjustme2 · 2018-02-22 · in Show HN: Headless Chrome Crawler -
Not a user of this tool, but https://github.com/yujiosaka/headless-chrome-crawler#event-n... points to https://github.com/GoogleChrome/puppeteer/blob/master/docs/a... where you can grab elements. Basically, when new page event happens, you…
comment by diggan · 2018-02-22 · in Show HN: Headless Chrome Crawler -
▲ 172
Show HN: Headless Chrome Crawlerstory by yujiosaka · 2018-02-22 · 33 comments