Tool directory
Every scraping API, crawler framework, and AI web agent we track, in one place.
Code you can read, fork, and run yourself. Some are backed by companies, others by a maintainer's spare weekends; the license is only half the story.
Mature Python framework for building and running your own crawlers
Node and Python crawling library with built-in anti-blocking
