webscrape.dev

Chrome's Headless Rewrite Changed the Arithmetic of Your Own Browser Fleet

Running headless Chrome at crawl scale now means running something much closer to a full browser. A look at what that did to per-session cost and when a non-Chromium engine or a rented fleet wins.

Nathan Kessler

Written by Nathan Kessler

Last updated: 7 min read

Chrome's Headless Rewrite Changed the Arithmetic of Your Own Browser Fleet

Chrome 132 reached stable on 14 January 2025, per the Chromium Dash milestone schedule. The line in it that matters to anyone operating a crawler reads like a footnote in Chrome's headless documentation: "Since Chrome 132.0.6793.0 the old Headless mode is only available as a standalone binary named chrome-headless-shell."

Nothing was deleted. Google split the old implementation out rather than dropping it, and had been shipping chrome-headless-shell as a separate download since Chrome 120, per its announcement post dated 6 March 2024. What went away was the default. The thing you now spin up per page when you pass --headless is the same engine a person runs on a laptop.

What the old mode actually was

Two different programs have gone by the name "headless Chrome", so it is worth being exact about which one got replaced. Chrome's own documentation describes the original as "a separate, alternate browser implementation that happened to be shipped as part of the same Chrome binary. It didn't share any of the Chrome browser code." The announcement post calls the shell "a lightweight wrapper around Chromium's //content module" with "substantially fewer dependencies," one that needs neither X11 or Wayland nor D-Bus.

Those absent dependencies were the feature. They were also why it behaved differently from real Chrome often enough to fail on defended sites. Google framed the split as "a trade-off between performance and authenticity," which is an honest description and also a cost statement.

Most stacks crossed this line earlier and more quietly, at a dependency bump. Puppeteer's docs are blunt: "Before v22, Puppeteer launched the old Headless mode by default." A team that upgraded past v22 changed which browser its crawlers run without touching crawler code. The shell is still reachable through headless: 'shell'. Playwright manages its own browser builds, so in either case the thing to check is what your driver launches, not what Chrome stable happens to ship.

Where the cost lands

Per-page cost in a browser fleet has three parts: process startup, resident memory while the page is open, and the CPU of running the page. The third is mostly a property of the page and barely moved. The first two are properties of the binary, and they did.

Concurrency on a worker is almost always bounded by memory rather than cores. If per-browser footprint rises by some factor, the number of concurrent sessions a machine holds falls by roughly that factor, and instance count rises to match. At a few thousand renders a day this sits inside the noise. At a few million it is the line someone circles in the infrastructure review, without a single thing about the crawl having changed.

I have no measurements of my own to put here, and would distrust anyone offering theirs without the harness. The most specific public figures come from a vendor with an obvious interest in them. Lightpanda, a headless engine written in Zig rather than forked from Chromium, publishes a comparison claiming a 123 MB memory peak against Chrome's 2 GB and 5 seconds against 46, for chromedp fetching 933 real pages on an AWS EC2 m5.large. That is the vendor benchmarking its own product on a workload it chose, not an audited result, and it read that way on its site on 27 July 2026. The direction is consistent with the architecture, since an engine that executes JavaScript and draws nothing pays for no compositor and no GPU process. Treat the direction as informative and the multiples as marketing until someone reproduces them. Our guide to reading scraping benchmarks covers why that distinction bites harder in this category than most.

The rented fleets do not meter the same thing

Once your own fleet gets heavier, renting is worth pricing. The complication is that browser infrastructure vendors meter different quantities, so their pricing pages do not produce the comparison you want.

Browserbase sells browser hours. Read on 27 July 2026, its Developer plan lists at $20/month with 100 browser hours included and $0.12 per hour after, and its Startup plan at $99/month with 500 hours and $0.10 per hour after, at 100 concurrent browsers.

Browserless sells units, defined on the same date as "a block of browser time of up to 30 seconds per browser connection," with another unit consumed for every further 30 seconds. Annual-billed tiers list at $25/month for 20k units, $140/month for 180k units with 50 concurrent, and $350/month for 500k units with 120 concurrent. Proxy bandwidth is billed in units on top, a listed 6 units per MB residential and 2 per MB datacenter.

Convert and the shapes diverge. 180,000 blocks of 30 seconds is 1,500 hours of ceiling, which prices out near $0.09 per fully saturated hour and looks cheaper than a $0.10 hourly rate. But a unit is charged per connection, not per second used. A page that renders in four seconds still burns a whole unit. At that duration the same plan buys 180,000 renders, roughly 200 hours of real work, and the effective rate lands closer to $0.70 per working hour. Block metering punishes short renders. Hourly metering punishes idle sessions and rewards holding one session across many pages.

Neither is a trick. They are billing models fitted to different usage shapes, and which is cheaper depends on a fact about your crawl that only you hold: how long a page keeps a browser open. Run either model at a million renders a day and you also learn something else, which is that published tiers stop well short of you and the conversation becomes a contract.

Steel is a useful middle case, listing a $250/month tier with $100 of monthly credits while also shipping an open-source Docker image, so the same API runs on your cluster or theirs and the build-versus-buy line stays movable. Hyperbrowser and BrowserCloud sit on the managed side of the same category. Read each metering definition on the vendor's own page rather than trusting a number quoted second-hand, this post included. Prices here move. The longer version of that decision is in our guide on self-hosted versus managed browsers.

The levers, in rough order of payoff

Render only what needs rendering. The cheapest session is the one that never starts. Fetch first, parse, and escalate to a browser only when the parse comes back empty or the response carries an interstitial. On most target sets that routes the majority of URLs away from the expensive path, and teams running a crawling framework already have the hook for it in their request pipeline. The failure mode is an escalation rule nobody revisits, so a site that stopped needing JavaScript in 2024 is still being rendered today.

Reuse contexts, not processes. Startup is the part of the bill that grew most when the binary became the whole browser. A pool of long-lived browsers handing out fresh contexts per job amortises that across hundreds of pages. The constraint is that a context is not a clean browser: fingerprint surface and anything cached above the context boundary carry over, which matters against an anti-bot system that scores consistency across a session. The cheap thing and the undetectable thing pull against each other, as covered in why scrapers get blocked.

Use a lighter engine where the DOM is all you need. Some share of any target set needs JavaScript to execute but needs no pixels, fonts, or compositor. That is where a non-Chromium engine, or chrome-headless-shell itself, earns a lane. The catch is compatibility, and it is not small: Google made the full browser the default precisely because the light path does not behave identically, and on a defended target a behavioural difference is a detection surface. Anything running against a hardened site belongs in the full-browser lane, which is the argument in browser agents for hard targets.

Why this ends as a split fleet

The tempting move is to pick the cheap engine, move everything over, and book the saving. Target sets are rarely uniform enough for that. A crawl across ten thousand domains contains static pages that never needed a browser, JavaScript-heavy pages that need a DOM and nothing more, and a defended minority where authenticity is the product and a lighter binary is a liability.

So the work this quarter is unglamorous.

Find out which browser your crawlers actually launch. A Puppeteer or Chrome upgrade may have answered that for you already, and the answer determines whether the memory line you are paying is the shell's or full Chrome's.

Instrument browser-seconds per successful extraction rather than requests per minute. Both billing models price wall clock, and almost nobody watches it. That single number decides whether unit-based or hour-based metering is cheaper for you, and no vendor page can supply it.

Then sample a few hundred URLs from your real target set and label how each one actually needs to be fetched: no browser, DOM only, or full Chrome. Those labels are the fleet design. The sums in this post, and every other cost argument you will read, only resolve once you have them.

Further reading: the web scraping stack in 2026 on where these layers sit relative to each other, and large-scale crawl architecture on the scheduling side of the same problem.

Share:

Tags:

  • #browser-infrastructure
  • #engineering
  • #frameworks
  • #cost