LLM Extraction or CSS Selectors: Decide Per Field, Not Per Project

ScrapeGraphAI charges 1 credit to fetch a page as clean markdown and 5 to run a prompt over it. That gap, published on its own pricing page and read on 2026-07-27, is the part of the AI-extraction argument that shows up on an invoice: fetching is close to free, the model pass is five times the fetch, and you pay it again on every page you send.

Most writing about this treats it as a product choice. One camp says prompt-based extraction removes the maintenance burden of selectors, so selectors are legacy. The other says a model is a probabilistic parser billed by the token, so anyone using one is paying for a regex they were too lazy to write. Both positions hold up in isolation and both fall apart the moment you look at an actual crawl, because an actual crawl is not one extraction problem. It is fifteen fields with fifteen different stability profiles running against a target set with a long tail.

The useful unit of the decision is the field, not the project. Once you make that switch the argument mostly dissolves, and what is left is arithmetic.

The framing problem

Most published comparisons of AI scraping and traditional scraping present them as two products you pick between. The genre is familiar: a table with rows for maintenance, accuracy and ease of use, a verdict that AI is the future, and no row anywhere for cost per field per page. That omission matters, because cost per field per page is the only axis on which the two approaches differ by orders of magnitude rather than by degree.

A CSS selector has a fixed cost you pay once, in engineering time, and a marginal cost of approximately zero. Running soup.select_one("span.price") across ten million pages costs the same per page as running it across ten. It breaks when the site changes, and then you pay the fixed cost again.

An LLM call has a small fixed cost, since a prompt takes minutes to write, and a marginal cost that is real and recurring. It survives site changes that would break a selector, and it charges you for that resilience on every row, forever.

Neither curve is better in the abstract. They cross at some volume, and where they cross depends on the field.

The field-level test

Ask four questions of each field you want to extract.

Does it sit in the same place on every page? If the price is always in .product-price across the crawl, that is a selector. There is no scenario in which paying a model to re-derive that fact on every page is a good trade.

Is it uniquely selectable? Some fields are stable in position but not addressable. A spec sheet rendered as an undifferentiated <div> of text with no per-row markup has no selector for "warranty period" even though the value is always in the same paragraph. That is a model's job.

Does the value have to be inferred? "Is this job listing remote-friendly" is not a field on the page. It is a conclusion drawn from a location string, a benefits list, and sometimes a sentence buried in the description. No selector exists for a judgement. If you want that column you are paying a model for it, and the only question left is how few tokens you can get away with sending.

How many distinct layouts does this field appear across? This is the question that decides most cases. Extracting a price from one retailer is a selector. Extracting a price from four hundred independent retailers, each with their own template, is four hundred selectors to write and maintain, and that is where a model starts winning on total cost even at meaningful volume.

Put together, the rule reads: selectors for stable fields on templates you crawl repeatedly, models for variable fields and for the long tail. Most pipelines have both, and the teams with costs under control are the ones that route field by field rather than picking a camp.

What each approach actually costs

The failure-mode row is the one people underweight. A broken selector throws. A confused model returns a well-formed string that is wrong, and if that string is a SKU or a price it enters your database with the same confidence as every correct row. So put a validator on any field where a plausible wrong answer does more damage than a missing one, and stop worrying about the rest.

The latency row matters for a different reason. If you are already fighting anti-bot systems on a constrained concurrency budget, adding a multi-second model call inside the fetch loop lowers your throughput ceiling. Extract asynchronously from stored HTML wherever the pipeline allows it.

The token arithmetic

Once you accept that a model bills per page, the size of what you send it becomes a cost decision rather than a formatting one.

Raw HTML is mostly not content. The Dripper paper (arXiv 2511.23119, "Token-Efficient Main HTML Extraction with a Lightweight LM") states that its analysis of 14,000 Common Crawl files shows 29.3 percent of pages exceeded 32k tokens and 21.0 percent surpassed 128k tokens. Those are not exotic pages. That is the modern web, where a product page ships a framework runtime, an analytics bundle, inlined critical CSS and a cookie consent modal alongside the four values you wanted.

Cleaning helps a great deal. How much is a question I would not answer with a single number: published reduction figures move around by vendor and by target, and the honest instruction is to measure a sample of your own pages before you plan a budget around someone else's percentage. The direction is not in dispute, and neither is the size of the prize, which is what the Dripper distribution above is really telling you.

Two practical consequences follow.

Strip before you send, always. Boilerplate removal is the cheapest optimisation in the pipeline, cheaper than switching models and far cheaper than renegotiating a plan. Every extraction tool worth using does it by default, which is why Firecrawl built its reputation on markdown output rather than on the extraction step layered above it.

Send fragments, not pages. If you know the review block lives inside #reviews, select that subtree deterministically and send only that. This is the hybrid pattern in miniature, and it routinely cuts the input by another order of magnitude beyond markdown conversion. You are using a selector for the part of the job selectors are exact at, locating a region, and the model for the part they cannot do, reading unstructured content inside it.

Input format also affects quality, not just price. The NEXT-EVAL paper (arXiv 2505.17125, May 2025) evaluated off-the-shelf models across several input preparations and reports that flat JSON produced the best extraction accuracy, an F1 of 0.9567, with minimal hallucination compared to the other formats it tested, ahead of slimmed HTML and hierarchical JSON. Copying that exact format matters less than the ranking behind it: preprocessing moves accuracy more than model choice does, and it is the part most teams never tune.

What vendors charge for the model pass

Here is the part the AI-versus-traditional comparisons skip. The extraction step is separately metered by most vendors, and it is worth planning around.

ScrapeGraphAI publishes the clearest rate card, which is where the 1-versus-5 figure at the top of this guide comes from. Its extract call, described as structured data from a natural-language prompt, carries a stealth modifier on top of the 5 credits. Search costs 2 credits per result without a prompt and 5 with one, the same multiplier showing up in a second product. Plans on that page, read on 2026-07-27, run Free at 500 one-time credits, Starter at $20 a month for 10,000, Growth at $100 for 100,000 and Pro at $500 for 750,000.

Firecrawl meters the same way but does not publish the multiplier. Its pricing page on 2026-07-27 lists a scrape at 1 credit per page and says only that "Advanced features (JSON format, Enhanced Mode, etc.) cost additional credits", without stating how many. Its plans on that date ran Free at 1,000 credits, then Hobby at $16 a month billed yearly for 5,000, Standard at $83 for 100,000, Growth at $333 for 500,000 and Scale at $599 for 1,000,000. If you are sizing a Firecrawl migration, the per-page scrape number is safe to plan with and the JSON-mode surcharge is not, so meter a real sample against your own account before you commit.

I could not confirm current published credit costs for the model pass at Diffbot, Thunderbit or Parsera from their own documentation within this guide's research budget, so I am describing the shape of each rather than quoting numbers I cannot stand behind.

Diffbot is the informative structural exception. Its extraction runs on purpose-trained models applied to a fixed ontology of articles, products, discussions and organisations, rather than a general LLM invoked against your schema per request. That makes it wrong for you if you need bespoke fields on an unusual page type, since you get its ontology rather than yours. It is right when your fields fall inside that ontology, because you are then buying a per-page extraction product rather than renting a per-token one.

Thunderbit meters on a different axis again, billing by output row from a browser extension aimed at operations and sales teams. Row billing suits that audience and makes small jobs predictable. Engineers who want an API-first pipeline will not enjoy it, and the bill climbs fast on high-cardinality pages where one URL yields hundreds of rows.

Prices move constantly in this category. Every figure above is dated to 2026-07-27 and attributed to the vendor's own page. The multipliers are more durable than the dollar amounts, and both should be re-checked before you budget.

The hybrid pattern

Teams running extraction at volume tend to arrive at the same architecture whether or not they set out to. It has four stages.

Fetch deterministically. The model should never be in the fetch path. Use whatever your target difficulty requires, from a plain HTTP client to a full browser infrastructure provider, and get the HTML into storage. That layer is a separate problem with its own economics, covered in the guide to choosing a scraping tool and, for hard targets, in why scrapers get blocked.

Strip and localise before the model sees anything. Convert to markdown or clean text, and where you can, narrow to the relevant subtree with a selector first. Every token you fail to remove here is billed forever.

Run the model only on the ambiguous fields. Not on the page. On the fields that failed the four-question test. If eleven of your fifteen fields have stable selectors, eleven of them should never touch a model, and the prompt should ask for four values, not fifteen.

Cache the result back into a selector where the target is a template. This is the stage most teams skip, and it is where the largest saving sits.

The published evidence for that fourth stage is stronger than most people realise. The WebLists paper (arXiv 2504.12682, April 2025) introduced a benchmark of 200 data-extraction tasks across four business and enterprise use cases, and reports that LLMs with search capabilities managed 3 percent recall and state-of-the-art web agents 31 percent. The system the authors propose, BardeenAgent, converts an agent's execution into a repeatable program: it constructs a generalisable CSS selector to capture all relevant items on the page, then fits the operations that extract the data, and replays that across pages with similar structure. They report 66 percent recall and a 3x reduction in cost per output row. The agent shares its name with a commercial automation product, and I am not aware of an independent replication, so treat the headline numbers as the authors' own. The structural claim is the part worth taking: applying a model to a sample and a selector to the population beat applying a model to everything, on both cost and recall.

Several vendors in the category market a version of this idea, generating a reusable scraper or selector from one model pass and then running subsequent pages cheaply. Parsera is the one usually named for it. I could not verify its current code-generation terms against its own documentation on 2026-07-27, so confirm the credit mechanics directly with the vendor before you architect around them. The pattern is sound regardless of who sells it, and you can build it yourself in an afternoon on any template site.

AgentQL approaches the same durability problem from the query side, with natural-language queries it describes as self-healing across layout changes, so the same query keeps working when a class name moves. That is a different bet: AgentQL pays the model repeatedly in exchange for never rewriting the query, where a cached-selector approach pays the model once in exchange for an artifact that can go stale. Which is right depends on how often your targets actually change, and most teams have never measured that.

For teams that want to own the loop rather than buy it, llm-scraper is a library rather than a service. You bring your own model and schema and pay your provider directly, with no per-page vendor surcharge on top. It is wrong for anyone without a fetch layer already, since it does no unblocking or rendering of its own, and wrong for non-engineers entirely. It is right when token spend has grown large enough that a vendor's extraction multiplier is the dominant line item and you would rather optimise prompts against raw model pricing.

The selector half of the hybrid is still, in most Python pipelines, Beautiful Soup. It does no fetching and sits in the crawling frameworks category rather than the AI one, which is roughly why nobody writes about it. It is also what does the work on the eleven fields you correctly kept away from a model, at zero marginal cost. Any architecture that has removed it entirely has probably moved cost rather than eliminated it.

Fields that should never go to a model unguarded

Some fields fail loudly when a selector breaks and fail silently when a model is confused. Those are the ones to protect.

Identifiers. SKUs, ISBNs, ASINs, listing IDs, anything you join on. A hallucinated identifier does not error, it creates a phantom record.

Prices and quantities. Not because models are bad at reading numbers, but because currency, tax inclusion, unit, and strikethrough versus current are all context a model resolves by inference and a selector resolves by position. If you must use a model here, constrain the output type and validate the range.

Dates. Relative dates ("3 days ago") are genuinely a model's strength. Absolute dates in a known format are a selector plus a parser, and turning that into a model call adds cost and a failure mode for nothing.

URLs. An href is right there in the DOM. There is no version of this where asking a model to transcribe it is the correct engineering decision.

The general rule: if a wrong value is worse than a missing value, put a deterministic check behind whatever produced it. A regex on the identifier format and a range assertion on the price will do. It is cheap, it runs locally, and it converts the model's quiet failure mode into a loud one.

Choosing where to start

If you are already parsing a small number of stable templates and your selectors mostly work, AI extraction has little to offer you today. Your cost is engineering time on breakage, and the fix is better monitoring, not a token bill. Adding a model here is a cost increase dressed as modernisation.

If you are staring at a long tail of unrelated layouts and the selector count is what is blocking the project, this is the case the AI extraction category was built for, and the surcharge buys coverage you were never going to hand-write.

If you are somewhere in between, which is most teams, the sequence that works is: keep the selectors you have, route only the fields that genuinely vary to a model, strip aggressively before every model call, and revisit the highest-volume templates to see whether a cached selector can pull them back onto the free curve. That last step is where the production stack tends to settle once the first invoice arrives.

If the blocking problem turns out to be fetching rather than parsing, none of this is your bottleneck. Extraction economics only start to matter once you can reliably get the page, a separate decision covered in the comparison of web unblockers and scraping APIs. Teams without engineering capacity are usually better served by the no-code scrapers than by either half of this argument, and teams who want fetch and extraction behind one endpoint should look at the web scraping APIs first.

Bottom line

Decide per field. Keep the selector where the value is stable and uniquely addressable, and spend tokens where it moves, where it has to be inferred, or where the tail of layouts is long enough that hand-writing parsers is the actual bottleneck. Strip the input before every model call, since that is the cheapest lever you have. On any template you crawl repeatedly, treat the model as the thing that writes the parser once rather than the thing that is the parser.

Browse the AI extraction category when you know which fields you are handing over.