Concurrency Limit

A concurrency limit is the number of requests a provider will let you have in flight at the same moment. It is not a rate limit. A rate limit counts how many requests you may start per unit of time, usually per second or per minute, and it does not care how long each one takes. A concurrency limit counts how many are open right now and does not care how fast you opened them. A crawler can sit far under a 100 requests per second rate limit and still be hard-stopped at 20 concurrent connections, and the same crawler can hold well under 20 concurrent connections and still trip a per-minute rate limit during a burst.

Vendors enforce it in three different ways, and the difference matters more than the number. Some reject the excess outright, typically with HTTP 429 and sometimes with a Retry-After header. Some queue the excess server side and answer eventually, which never produces an error but inflates every latency number you record. Some simply refuse the TCP connection or reset it, which surfaces in your client as a connection error rather than an HTTP status and is easy to misclassify as a network fault. Read the vendor's own status code reference before you assume a 429 is coming. ScraperAPI's status code page for its AI parser endpoint, read 2026-07-27, documents 200, 202, 204, 400, 401, 402, 404 and 500 and mentions no 429 at all, a reminder that the documented surface is often narrower than the real one.

The throughput arithmetic is fixed and worth internalising: achievable requests per second equals concurrency divided by mean response time. Fifty concurrent slots against a target that averages 5 seconds per response gives 10 requests per second, not fifty. Buying more concurrency against a slow target buys you very little, because the constraint is the target's clock, not the contract. This is also why headroom on the limit is not proof of health. If you hold 50 slots open and only push 10 requests per second, you are not throttled, you are waiting.

In practice, size purchased concurrency against the target's observed p95 latency rather than against a throughput number you would like to hit, and re-measure after any change to rendering or proxy type, since both move latency substantially. When latency spikes, separate vendor-side queueing from a genuinely slow target by timing at two points: connection established to first byte tells you about the target, request issued to connection established tells you about the queue in front of it. If the second number grows while the first stays flat, you are sitting in the vendor's queue and more workers will not help.