Tool directory
Every scraping API, crawler framework, and AI web agent we track, in one place.
Run on your own infrastructure. You trade managed convenience for control over data, cost, and compliance.
Mature Python framework for building and running your own crawlers
Node and Python crawling library with built-in anti-blocking
