apify/crawlee

View on GitHub ↗ ★ 26,316 TypeScript

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

10 mentions on Hacker News by 8 people: 3 stories, 7 comments · first 2018-06-02 · latest 2025-10-03

Mentions per year 10 mentions · busiest year 2018 (3)

20082026

All Stories Comments Newest Top Oldest

▲ 254
Show HN: Crawlee for Python – a web scraping and browser automation library
story by jancurn · 2024-07-09 · 57 comments
▲ 2
Crawlee: Crawlee–build reliable crawlers. Works with Puppeteer, Playwright, Ch
story by todsacerdoti · 2024-05-24 · 0 comments
▲ 78
Show HN: Apify SDK – A scalable web crawling and scraping library for JavaScript
story by jancurn · 2018-09-25 · 8 comments