← All repos

google/robotstxt

View on GitHub ↗ ★ 3,479 C++

The repository contains Google's robots.txt parser and matcher as a C++ library (compliant to C++11).

8 mentions on Hacker News by 7 people: 0 stories, 8 comments · first 2019-07-01 · latest 2025-03-18