The directory for web scraping and AI data tools

Bright Data

Large proxy network with scraping APIs for enterprise data

Oxylabs

Proxy pools and scraper APIs built for large-scale data teams

ScraperAPI

Handles proxies, browsers, and CAPTCHAs behind one endpoint

Zyte

Scraping API and cloud platform from the team behind Scrapy

Scrapy

Mature Python framework for building and running your own crawlers

Crawlee

Node and Python crawling library with built-in anti-blocking

Decodo

All-in-one proxy platform, rebranded from Smartproxy in 2025

SOAX

Residential, mobile, and datacenter proxies with a data API

IPRoyal

Pay-as-you-go proxy network with a web unblocker API

Nimble

Proxy infrastructure with AI browser rendering and data APIs

Infatica

Residential and mobile proxy network with scraping APIs

Massive

Residential proxy network with Web Access, Render and Search APIs

ProxyEmpire

Residential, mobile, and datacenter proxy network for web scraping

Evomi

Swiss residential and datacenter proxy network with a scraper API

IPBurger

Residential, mobile, ISP, and dedicated proxy network

ProxyScrape

Residential, datacenter, ISP, and mobile proxy network plus SERP API

PacketStream

Peer-to-peer residential proxies at $1 per GB, no minimum spend

SimplyNode

Residential and mobile proxy network with pay-as-you-go pricing

Geonode

Proxy network and scraping API with flat per-request pricing

ProxyRack

Residential, datacenter, and mobile proxies across 140+ countries

Shifter

Residential and ISP proxy network with scraping and SERP APIs

Proxy-Cheap

Residential, datacenter, and mobile proxy network with per-GB pricing

DataImpulse

Pay-as-you-go residential, datacenter, and mobile proxies

Froxy

Proxy network with SERP and e-commerce scraping APIs

Byteful (formerly Ping Proxies)

Proxy network provider with four proxy types and a REST API

Mars Proxies

Proxy network with residential, datacenter, ISP, and mobile IPs

Proxidize

Managed mobile and residential proxies with a cloud dashboard and API

Live Proxies

Residential and mobile proxy provider with private IP allocation

GoProxies

Residential, ISP, and datacenter proxy network with 80M+ IPs

Rayobyte

Proxy network with residential, ISP, and datacenter IPs

Webshare

Datacenter, ISP, and residential proxies with a free tier

Proxy-Seller

Sells residential, mobile, ISP and datacenter proxies

NodeMaven

Residential and mobile proxy network with ZIP-level targeting

Browse AI

No-code tool for training robots to scrape and monitor websites

Octoparse

No-code scraper that turns web pages into structured data

Apify

Cloud platform for running and sharing scraping bots

ParseHub

Desktop point-and-click scraper for JS-heavy sites, exports JSON/CSV

Bardeen

No-code browser extension for web scraping and lead research

Simplescraper

Chrome extension and cloud service for no-code scraping recipes

Import.io

Managed and no-code web data extraction with retail pricing tools

Stabler

Node-graph builder for JS-rendered scrapers with anti-bot bypass

No-Code Scraper

No-code AI scraper that exports data to CSV, Excel, or JSON

Dexi.io

No-code platform for e-commerce price and assortment monitoring

PhantomBuster

Cloud-based no-code tool for scraping and automating social platforms

Web Scraper

Free Chrome extension for visual scraping, paid cloud platform

Databar.ai

No-code GTM data enrichment via 160+ third-party providers

ScrapeStorm

Desktop visual scraper with AI field detection and flowchart mode

Roborabbit

No-code platform for browser automation and web data extraction

Hexomatic

Cloud-based no-code scraper with 100+ prebuilt automations

Lobstr.io

No-code cloud platform for prebuilt LinkedIn and Maps scrapers

Listly

Point-and-click Chrome extension for scraping data into spreadsheets

Automa

Open-source browser extension for no-code workflow automation

Mozenda

Point-and-click web scraper with a managed-service option

Axiom.ai

No-code Chrome extension for browser automation and scraping

WebHarvy

Windows desktop app for visual, no-code web scraping

Instant Data Scraper

Free browser extension that auto-detects and exports web page tables

Data Miner

Chrome/Edge extension for point-and-click scraping to CSV/Excel

ZenRows

Scraping API with proxy rotation, anti-bot bypass, and JS rendering

Scrapeless

Cloud scraping API and proxy network billed only on success

ScrapingAnt

Web scraping API with headless Chrome rendering and rotating proxies

Scrapingdog

Web scraping API with rotating proxies and JS rendering

Scrapfly

API-first scraping platform with anti-bot bypass and proxy rotation

WebScrapingAPI

Hosted scraping API with JS rendering, SERP and Amazon parsers

Scraping Fish

Web scraping API with rotating proxies and headless JS rendering

ScrapeNinja

Web scraping API with proxy rotation and Chrome fingerprinting

ScrapeOwl

Web scraping API with proxy rotation and CAPTCHA bypass

Scrappey

Pay-per-request scraping API with residential proxies, JS rendering

Scrapestack

REST web scraping API with proxy rotation and JS rendering

WebScraping.AI

Web scraping API with headless Chrome rendering and AI extraction

Olostep

Web data API with built-in JS rendering, proxies, and CAPTCHA solving

HasData

Managed API turning any URL into structured JSON or Markdown data

Scraping Bot

Scraping API with prebuilt endpoints for e-commerce and real estate

Scrape.do

Scraping API with rotating proxies and ready-made site endpoints

Crawlbase

Web scraping API platform with JS rendering and residential proxies

Outscraper

Pay-as-you-go scraper APIs for Google Maps and business data

Zenscrape

Web scraping API with proxy rotation and headless rendering

Scrapin

API for structured LinkedIn-sourced professional and company data

ScrapingBee

Scraping API with headless Chrome rendering and AI extraction

ScrapeOps

Aggregates 20+ proxy providers with a scraper ops dashboard

Firecrawl

Open-source API that scrapes and crawls sites into LLM-ready data

DataFuel

API that turns websites into markdown or JSON for RAG and LLMs

Expand.ai

Turns websites into type-safe APIs using AI-generated schemas

ScrapeGraphAI

AI-powered API that turns any webpage into structured JSON via prompts

Parsera

No-code AI scraper that turns prompts into structured-data APIs

LLM Scraper

TypeScript library that turns webpages into structured data via LLMs

Kadoa

AI platform for extracting and monitoring structured web data

AgentQL

Natural-language query language for structured web data extraction

Diffbot

Computer vision and ML API that extracts structured web data

Jina Reader

Turns any URL into clean markdown via a simple API prefix

Thunderbit

Browser extension using AI prompts to extract web page data

Exa

Semantic search API for AI agents, returning structured JSON

BrowserOS

Open-source Chromium browser with a built-in AI agent

JigsawStack

Small AI models via API for web scraping, search, and OCR

Parsio

SaaS parser that extracts structured data from emails and documents

Linkup

Search, fetch, and research API for AI applications

FetchFox

Open-source AI scraper with a hosted API and dashboard

Tavily

Real-time web search, extract, and crawl API for AI agents

Kameleo

Self-hosted anti-detect browser with spoofed device fingerprints

Multilogin

Antidetect browser platform with cloud phones and residential proxies

Octo Browser

Antidetect browser for multi-account profiles with proxies and API

Incogniton

Anti-detect browser for running multiple fingerprinted profiles

GoLogin

Antidetect browser for isolated profiles with spoofed fingerprint

Dolphin Anty

Chromium antidetect browser for multi-accounting and scraping

AdsPower

Antidetect browser for running isolated, fingerprint-spoofed profiles

Hidemyacc

Antidetect browser with isolated profiles and proxy support

MoreLogin

Antidetect browser with cloud Android devices for account management

Nstbrowser

Cloud antidetect browser bundling proxies and CAPTCHA solving

RoxyBrowser

Antidetect browser for managing multi-account profiles

GeeLark

Cloud phone platform for managing multiple social accounts

Linken Sphere

Desktop antidetect browser for running isolated multi-account profiles

Undetectable

Antidetect browser for fingerprint masking and multi-accounting

Hidemium

Antidetect browser for running many isolated profiles

Stagehand

Open source AI browser automation SDK from Browserbase

Browser Use

Open-source browser agent framework plus a paid cloud API

Skyvern

Open source browser automation agent using LLMs and vision

Nanobrowser

Open-source Chrome extension for multi-agent browser automation

Bytebot

Open-source AI agent that automates tasks via a virtual desktop

CloudCruise

API platform for browser automation agents in healthcare workflows

rtrvr.ai (Retriever AI)

Browser-extension AI agent for automating tasks and scraping data

Agent-E

Open-source LLM agent framework for browser automation

Asteroid

AI browser agents for healthcare portals and EHR workflows

Notte

Browser automation platform with AI agents and structured scraping

LaVague

Open-source framework for building AI agents to navigate websites

Gumloop

No-code AI agent builder with web scraping and crawling nodes

TinyFish

Unified API platform giving AI agents structured access to the web

Cua

Open-source framework for AI agents to control desktop OS sandboxes

Colly

Open-source Go library for building web crawlers and scrapers

Playwright

Open-source browser automation library for Chromium, Firefox, WebKit

Crawl4AI

Open-source Python crawler built for LLM and AI-agent pipelines

Spider

A Rust-based web crawling and scraping API with JS rendering

Katana

Open-source Go web crawler with headless JS rendering and JSON output

Puppeteer

Node.js library from Google for headless Chrome automation

Selenium

Open-source browser automation framework for testing and scraping

Beautiful Soup

Open-source Python library for parsing HTML/XML documents

node-crawler

Node.js crawling library with connection pooling and Cheerio parsing

Nokogiri

Ruby library for parsing and querying XML and HTML documents

Apache Nutch

Java web crawler framework for large-scale, Hadoop-based crawling

SerpApi

JSON API for scraping Google and 50+ other search engines

SERPHouse

Real-time search results API for Google, Bing, and Yahoo

Serper

Google Search API with structured JSON results in 1-2 seconds

DataForSEO

Pay-as-you-go REST APIs for SERP, keyword, and backlink data

OpenWeb Ninja

Pay-per-request platform bundling 40+ SERP and web data APIs

SearchApi

SERP scraping API returning structured JSON for Google and more

Aves API

SERP scraping API returning real-time JSON or HTML results

Zenserp

SERP API for Google, Bing, Yandex and YouTube search results

SerpWow

SERP API for structured Google search data as JSON or CSV

Scale SERP

Google search results API with structured JSON output

Value SERP

Google SERP scraping API with JSON, HTML, or CSV output

Brave Search API

Search API over Brave's independent index, with an LLM endpoint

WebVoyager

Open-source multimodal web agent with a 643-task benchmark

WebArena

Self-hosted, open-source benchmark for autonomous web agents

Mind2Web

Benchmark of 2,350 tasks across 137 real websites for web agents

BrowserGym

Gym-style API unifying web-agent benchmarks like WebArena

MiniWoB++

Benchmark suite of 100+ simulated web tasks for web agents

AssistantBench

Academic benchmark testing AI agents on realistic web tasks

WebCanvas

Open-source benchmark for evaluating web agents online

GAIA

Benchmark for evaluating general AI agents on reasoning and tool use

Bananalyzer

Open-source Playwright benchmark for evaluating AI web agents

OSWorld

Benchmark for evaluating computer-use AI agents on real tasks

VisualWebArena

Benchmark of 910 visually grounded tasks for multimodal web agents

WebShop

Simulated e-commerce benchmark for training web-shopping agents

Browserbase

Managed headless-browser API for AI agents and developers

Browserless

Cloud headless browsers you drive with Puppeteer or Playwright

Lightpanda

Headless browser engine built in Zig, not a Chromium fork

BrowserCloud

Cloud browser automation platform for scraping and AI agents

Hyperbrowser

Managed cloud browsers for AI agents with scrape and crawl APIs

Surfsky

Cloud antidetect browsers with proxy management and captcha solving

Steel

Open-source cloud browser API with sessions and stealth for AI agents

Anchor Browser

Cloud browser infrastructure for AI agents to act on websites

Airtop

Cloud browsers that let AI agents log in and act on websites

People Data Labs

B2B API for person, company, and job-posting data enrichment

Datarade

Marketplace connecting data buyers with 2,600+ vetted providers

Coresignal

Structured company, employee, and job data via API and bulk files

Bright Data Datasets

Marketplace of 350+ pre-built and custom scraped datasets