Running Two Proxy Vendors: Failover, Cost Routing and Avoiding Lock-In

Oxylabs counts sticky-session lifetime in whole minutes and caps it at 1440. IPRoyal takes a single-unit duration string that runs from one second to seven days, and puts it in the password field rather than the username. Both vendors sell the same thing, and neither will accept the other's way of asking for it. That gap, rather than the proxy connection itself, is what turns a second provider into an engineering project.

Most teams arrive at the second provider by accident anyway. They sign with one vendor, wire the credentials into a config file, and ship. That works until the vendor has a bad afternoon, when every scraper in the fleet fails at the same instant because they all share one dependency. The fix then gets improvised under pressure: a second account opened that week, a hardcoded fallback, an environment variable nobody documents.

This guide covers the version you would build on purpose. The proxy network buying decision is covered in the guide to choosing a scraping tool; what follows assumes you have already picked two and need them to coexist.

One thing shapes everything below. Every vendor named here sells the same primitive: a plain HTTP or SOCKS gateway that takes credentials in the username or password field. Oxylabs, Decodo and IPRoyal all document that gateway in exactly those terms in the docs I read on 27 July 2026. The raw pool is the standardised part of this market, and it is the only part that is.

One vendor is one failure domain

A second provider is worth having even when the first one is reliable, because a single-vendor setup has no failure mode between "fine" and "everything stops."

Three things fail independently and none of them are the same event. The vendor's network has an incident, which is rare and usually short. The vendor's relationship with your target changes, which is neither rare nor short: a pool that worked against a site last month can start returning blocks this month because the target added the vendor's ASNs to a list, and no amount of retrying fixes it. And the commercial relationship changes, which includes price increases, a plan restructure, and a compliance review that decides your use case needs a different contract.

The second failure is the interesting one, and it is the reason failover cannot be a simple health check. When a pool starts failing on one target and nothing else, the pool is not down. If your gateway marks it down, you have just halved your capacity because of one hostile site.

The counter-argument is real and worth stating. A second vendor doubles your surface area for credential leaks, adds a second invoice to reconcile, and introduces routing code that can itself be the thing that breaks at 3am. Teams whose collection is not business-critical, or whose targets are cooperative, should not build this. The honest test is whether an eight-hour proxy outage would cost more than the routing layer costs to build and maintain.

The gateway is smaller than you think

The component in the middle is a local HTTP proxy that your scrapers point at instead of pointing at a vendor. It does five things and should do nothing else:

1. Accept a request with an internal pool hint and an internal session ID.
2. Pick a pool.
3. Rewrite the credential into that vendor's syntax.
4. Forward the request through that vendor's gateway.
5. Record the outcome against pool, target domain and session.

That is a few hundred lines in most languages, because the vendors did the hard part already. Each one wants to be a drop-in replacement for whatever proxy you are currently pointing at, so each publishes an endpoint that behaves like an ordinary HTTP proxy and a credential that carries the configuration. The vendors advertise this because it shortens their sales cycle. The side effect is that a product designed to slot into your code with one config change also slots in behind a router with one config change.

Two design choices matter more than the code.

Your scrapers must not know vendor names. They ask for a pool class (cheap, residential-us, mobile) and pass an opaque session ID. The moment a scraper contains the string brd-customer or gate.decodo.com, the abstraction has leaked and the migration you were protecting yourself against just got more expensive.

The gateway is on the request path, so it must be dumber than the thing it protects. No database call per request, no external config fetch, no clever retry logic that turns one blocked request into nine. Health state lives in memory, is updated from outcomes you are already collecting, and survives a restart badly on purpose: a cold gateway that routes naively for thirty seconds is better than one that fails to start because it cannot reach a state store.

Health checks that tell "dead pool" from "hard target"

The naive check is a request to a known-good URL through each pool on a timer. It answers exactly one question: is this vendor's gateway accepting connections. That is the failure mode you will almost never see.

What you need is a two-dimensional score: pool by target domain. Keep a rolling window per pair, and decide with two independent thresholds.

• Pool-level health is the aggregate across all domains. If a pool is failing everywhere at once, the vendor has a problem. Take it out of rotation entirely.
• Pair-level health is one pool against one domain. If a pool fails on example.com and passes on everything else, the pair is bad and the pool is fine. Stop sending that domain to that pool and keep using it for everything else.

The distinction is the whole point of the exercise. Anti-bot systems are per-target by design; a pool going bad on one site is the normal case, not the emergency. If your gateway cannot express "this pool is fine except against this one target," it will either overreact and cut capacity or underreact and keep feeding requests to a pool that will never succeed there. The anti-bot system glossary entry covers why the block is target-specific rather than a property of the pool.

Classify outcomes rather than counting non-200s. A 403 with a challenge page, a 429, a redirect to an interstitial and a TCP timeout are four different signals. Timeouts and connection resets point at the pool. Challenge pages and 429s point at the pair. A 404 points at your own URL construction and should not touch either score.

On the cost of probing: residential proxy traffic is billed by the gigabyte at every vendor in this list, so a probe schedule is a real line item. Prefer passive scoring, where the requests you were sending anyway are the health signal. Reserve active probes for pools with no recent traffic, run them at low frequency, and point them at something small and static. Most vendors run an IP-echo endpoint for exactly this purpose; a request to one costs almost nothing and confirms the credential and the exit geography in a single round trip, though it tells you nothing about whether the pool can still pass a real target.

Routing by target class, then by price

Once two pools are healthy, the router has a choice to make on every request, and the two inputs are what the target demands and what the pool costs.

Target class first. Static pages behind no meaningful defence do not need residential IPs, and paying per-gigabyte residential rates for them is the most common way to overspend on proxies. Webshare is the obvious floor here: datacenter, ISP and residential tiers with a free entry point and published prices rather than a sales call, which makes it useful as the cheap default class even for teams whose real money goes elsewhere. It is the wrong choice as your only pool, because datacenter ranges are the first thing a serious anti-bot vendor filters.

Escalate by class, not by retry count. A request that fails twice on the cheap pool should be re-issued on the expensive one, and the target domain should then be marked as needing the expensive class for some window. Retrying the same class nine times is how a $200 month becomes a $2,000 month.

For the expensive class, the split is roughly between scale and flexibility. Bright Data and Oxylabs sit at the enterprise end with the largest pools and the most granular targeting, and both are heavier to procure than a small team wants; the Bright Data vs Oxylabs comparison covers where they diverge. Decodo, which rebranded from Smartproxy in 2025, and SOAX both offer freemium entry and are more practical as a second pool you can turn on this week without a procurement cycle. IPRoyal sells pay-as-you-go traffic, which makes it well suited to the low-volume expensive class, where you want capability available without a committed monthly spend. Massive is worth a look if part of your traffic is agent-driven rather than crawler-driven, since it ships Web Access, Render and Search APIs over the same network, though those APIs are exactly the kind of surface this guide warns about below.

Who each of these is wrong for: Bright Data and Oxylabs are wrong for a team that wants to start today without talking to sales. Webshare is wrong as a sole provider against defended targets. IPRoyal's pay-as-you-go model is wrong for predictable high volume, where committed pricing wins. Decodo and SOAX are wrong for a team that needs a named enterprise contract for compliance reasons.

A useful sanity check before you build any of this: price the same monthly volume across both pools at their published rates and see whether the spread justifies routing at all. If the cheap pool saves 15% and the routing layer costs a week, the routing layer is a reliability project, not a cost project, and should be argued for on those terms.

Sticky sessions do not survive a failover

This is where the abstraction stops being free. A sticky session is a promise that requests carrying the same identifier exit through the same IP for some period. Every vendor here supports it. No two express it the same way, and the differences are not cosmetic: they land in different fields, use different time units, and impose different constraints on the identifier itself.

Here is what the vendor documentation says. The Oxylabs, Decodo and IPRoyal rows were read against the vendors' own docs on 27 July 2026; the remaining rows reflect published documentation but were not re-read on that date, so treat them as a starting point and confirm against the vendor before you write the formatter.

Read the units column again. Oxylabs counts session time in whole minutes and caps it at 1440. IPRoyal takes a duration string with exactly one unit in it, anywhere from one second to seven days, so 10m is valid and 1h30m is not. Decodo runs from one minute to twenty-four hours. Three vendors, three ways of saying the same thing. A gateway that stores "session TTL" as a bare integer without a unit will eventually open a 300-minute session where it meant 300 seconds, and you will find out from the invoice rather than from a test.

IPRoyal is the one that breaks naive implementations, because its session and lifetime keys go in the password field rather than the username, alongside the country parameter. Any abstraction that assumes "session goes in the username" needs a per-vendor formatter, not a shared template with a variable in it. IPRoyal also requires the session identifier to be exactly eight alphanumeric characters, so your internal session IDs cannot be UUIDs unless the formatter truncates or hashes them down to eight. Webshare's numeric session identifiers impose a constraint from the other direction. The general lesson is that your internal session ID is not a string you can pass through; it is a value each formatter has to derive its own identifier from.

Even within one vendor the syntax forks by product. Oxylabs documents sessid in the username for its backconnect residential pool, while its dedicated datacenter and ISP proxies are separate products with their own setup pages, because those are per-IP rather than backconnect. A gateway keyed on vendor name alone will get this wrong; key it on vendor plus product.

The practical rule that falls out of all this: pin a logical session to the pool that opened it, and treat vendor failover as session termination. You cannot hand a session to another vendor, so do not build anything that pretends you can. When the pinned pool degrades, discard the session, discard the cookies and local state accumulated under it, and open a fresh session on the replacement pool. For a login-gated crawl that means re-authenticating. Design the scraper to tolerate that, because the alternative is a scraper that silently continues a "session" whose IP changed country mid-way, which most anti-bot systems treat as exactly the signal it is. The rotating proxy entry covers why per-request rotation avoids this problem entirely, which is a good argument for keeping sessions off any workload that does not strictly need them.

Two bills, two quotas, one accounting problem

Credentials multiply faster than vendors. A single provider can issue separate credentials for residential, datacenter, ISP and unblocker products, each with its own quota and its own username format. Two vendors can easily mean six live credential sets.

Credentials belong to the gateway, not to the scrapers. One secret store, read at gateway startup, never in a scraper's environment. This is a security argument first, but it is also what makes adding a third vendor a gateway deploy rather than a fleet-wide config change.

Attribute usage yourself. Vendor dashboards report bandwidth by their own sub-account structure, which will not match your job structure. If the gateway does not tag every request with a job or team identifier and log the byte counts, the first surprise invoice will be unattributable. Most vendors support sub-users or zones, and mapping one sub-user per major job is the cheapest way to make the vendor's own reporting line up with yours.

Quota exhaustion looks nothing like a block. A residential plan that runs out of prepaid traffic returns auth failures, not 403s. Health scoring keyed on request outcomes will read this as a total pool failure and correctly route away, but nobody will know why until someone opens the dashboard. Track remaining balance as a separate signal, alert on it, and remember that pay-as-you-go models like IPRoyal's fail differently from committed monthly plans.

What quietly makes a pool non-swappable

The raw proxy port is portable. Everything the vendors built on top of it is not, and the productised layers are where the margin is, which is why every vendor pushes you toward them.

Unblocker endpoints. Bright Data's Web Unlocker, Oxylabs' Web Unblocker, Decodo's Site Unblocker and IPRoyal's Web Unblocker all present as proxy endpoints, so they look swappable in a way the raw pools genuinely are. They are not equivalent. These products are configured through vendor-specific instruction surfaces: custom request headers at one vendor, zone configuration and username flags at another. Two products that both describe themselves as proxy-compatible unblockers still need different code to ask for the same behaviour, and the feature sets do not line up one to one. Treat any unblocker as a separate integration with its own migration cost, not as a pool you can swap at the gateway. Where a vendor publishes a migration guide from a competitor's unblocker, it is worth reading for the list of gaps it admits to, while remembering it is a sales document about a rival.

Prebuilt parsers and scraper APIs. A vendor endpoint that returns structured JSON for a search results page or a product listing is genuinely useful and genuinely sticky. The output schema is theirs. Every downstream consumer of that JSON is code you rewrite when you leave, and the rewrite is invisible in a per-gigabyte price comparison. If you use these, put an adapter between the vendor schema and your internal model on day one, while the mapping is still small.

Session and provisioning APIs. Endpoint generators, sub-user management APIs, IP-skip lists and saved browser profiles all live in the vendor's control plane. Orchestration built on them does not move.

None of this argues against using the productised layers. It argues for knowing which parts of your stack are swappable and which are not, and for keeping the swappable part genuinely swappable. In practice that means the raw pool sits behind the gateway and stays interchangeable, while unblockers and scraper APIs are treated as what they are: a web scraping API dependency with a migration cost, whether or not it happens to be sold by your proxy vendor.

When to buy the gateway instead of building it

The routing layer described here is not hard, but it is permanent. Somebody maintains it, gets paged when it misroutes, and keeps the seven session formats in the table above current as the vendors change them.

ScrapeOps sells that as a product. Its Proxy Aggregator page, read on 27 July 2026, advertises "over 20+ residential, mobile and smart API proxy providers from a single proxy endpoint," split across a residential aggregator and a separate aggregator for smart proxy APIs. One integration, one bill, one set of credentials, and the per-vendor session syntax becomes somebody else's problem.

The trade is straightforward. You give up per-request control over which provider handles which target, you accept a margin on top of the underlying proxy cost, and you add a dependency whose own availability now sits in front of every vendor you would otherwise reach directly. You also lose the ability to say which named provider handled a given request, which matters if that is a compliance requirement rather than a preference.

Aggregation is the right call for a team without an infrastructure engineer to spare, or one that wants multi-vendor resilience before it has multi-vendor volume. It is the wrong call for a team with negotiated rates it wants to exploit through routing, with target-specific pool knowledge worth encoding, or with a contractual need to name the provider. Between those poles, the deciding question is usually whether proxy routing is close enough to your product to be worth owning.

Bottom line

The router itself is a few hundred lines, because the raw proxy layer is standardised. What costs you time is the surrounding detail: health scoring that can tell a dead pool from a hostile target, sessions pinned to the pool that opened them, usage attributed centrally because no two vendor dashboards will match your job structure, and a clear line around the vendor features that sit above the proxy port, since those are the parts that make a pool non-swappable.

If none of that sounds worth owning, buy it. If it does, the blog post on the 2026 scraping stack covers where this layer sits relative to the crawling framework above it.