Three Companies, Three Core Web Vitals, Three Revenue Outcomes: What the LCP and INP Case Studies Actually Prove

The three case studies, stated precisely

Google publishes a Core Web Vitals case-study series on web.dev in which named companies document a performance change and the business metric that moved alongside it. Three of these are cited far more than the rest, usually stripped of their context and reduced to a single impressive percentage. Stated in full, they are more instructive than the headline versions.

Rakuten 24, a Japanese online store, optimized Largest Contentful Paint and reported a 53.37 percent increase in revenue per visitor and a 33.13 percent increase in conversion rate over the measured period. Vodafone Italy improved its LCP by 31 percent and reported an 8 percent increase in sales. redBus, an Indian bus-ticketing platform, improved Interaction to Next Paint, the metric that captures how quickly a page answers a tap, and reported a 7 percent increase in sales.

Every number in that paragraph is real, publicly documented, and attributable. Each also comes from exactly one company, measured before and after its own work, on its own traffic. That single fact governs everything that follows.

What a Core Web Vitals case study can prove, and what it cannot

A before-and-after case study is a record of one thing that happened once. It can establish that an outcome is possible. It cannot establish how often that outcome occurs, how large it is on average, or whether the change being credited actually caused it. In the language of evidence, these are existence proofs, not effect sizes.

The distinction matters because the numbers invite a specific error. When someone reads that Rakuten 24 gained 53.37 percent revenue per visitor, the tempting inference is that faster LCP yields something in the neighborhood of a 50 percent lift. That inference treats a single observation as an elasticity, a stable ratio between an input and an output that holds across cases. Nothing in a single before-and-after record supports that leap.

Why before-and-after inflates, in a direction we can name

The deeper point is not merely that a single case study is uncertain. It is that uncontrolled before-and-after measurement tends to overstate the effect of the thing being studied, and the mechanism is well documented in the measurement literature.

The strongest evidence comes from a large randomized field experiment at eBay. Blake, Nosko and Tadelis showed that observational estimates of paid-search return, the kind produced by ordinary attribution reporting, were inflated relative to the true causal effect measured experimentally, because ad clicks correlate with buyers who were already going to purchase. The general lesson generalizes past advertising: when the population you measure after an intervention is not a random slice, non-experimental estimates carry the pre-existing intent of that population, not the clean effect of the change. A company motivated enough to invest in a serious performance project, measured during the window it chose to report, is not a neutral sample.

Ron Kohavi, Diane Tang and Ya Xu, drawing on more than twenty thousand experiments a year at Microsoft, formalize the same caution as a working rule: a surprising result is usually an error until it survives a checklist of validity threats. Their canon exists precisely because uncontrolled or improperly controlled measurement produces confident numbers that do not replicate. A 53 percent figure from one company, with no control and no replication, is the archetype of a result their method would quarantine before trusting.

The large-N corroboration, and its own limits

If the three case studies were the only evidence, the fair verdict would be "interesting, unproven." They are not. A much larger study points the same way, which is why the direction of the effect is credible even though the magnitudes are not transferable.

"Milliseconds Make Millions," commissioned by Google and conducted by the agency 55 and Deloitte Digital, analyzed more than thirty million real user sessions across thirty-seven European and American brand sites. A 0.1 second improvement in mobile load speed was associated with an 8.4 percent increase in retail conversions and a 9.2 percent increase in average order value, with travel conversions rising 10.1 percent. That is a far stronger design than any single case study: many brands, many sessions, a consistent measured relationship.

It still is not a randomized controlled trial, and it was commissioned by a party with an interest in the finding, so it corroborates rather than proves. But it does the one thing the three case studies cannot do on their own: it establishes that the speed-to-revenue relationship holds across a population, not just in three flattering anecdotes. The case studies then become believable illustrations of a real pattern, which is exactly the weight they can bear and no more.

The tiers of evidence

Not all of the evidence in this article is equally strong, and treating it as if it were would repeat the error the article is about. It is worth sorting explicitly.

How to use these case studies without overclaiming

The case studies are useful the moment you stop asking them the wrong question. "How much will speed lift my revenue?" is unanswerable from this evidence. "Is it plausible that a slow page is costing me measurable revenue, and worth measuring on my own site?" is answered clearly: yes.

That reframes the work. The correct move is not to borrow Rakuten 24's 53 percent, or Vodafone's 8 percent, or redBus's 7 percent, and forecast it onto your business. It is to measure your own page against the thresholds Google actually reads, find where it fails, fix those specific faults, and observe your own before-and-after with clear eyes about its limits. Your own N of one is still an N of one, but it is measured on your traffic and your buyers, which makes it the only figure that describes you.

Google grades Core Web Vitals on field data at the seventy-fifth percentile of real visitors, against published thresholds: Largest Contentful Paint at or under 2.5 seconds, Interaction to Next Paint at or under 200 milliseconds, and Cumulative Layout Shift at or under 0.1. A page can look finished in the browser and still fail these checks for the slice of users on slower devices and networks. INP, the responsiveness metric redBus improved, is a common failure point because it captures the lag after a tap that a fast-looking page can still hide.

The rule this leaves you with

Three companies improved three Core Web Vitals and reported three revenue gains. The evidence is real, it is documented, and pointed in the same direction as a study of thirty million sessions. That is enough to justify taking page performance seriously as a revenue lever. It is not enough to promise a number, and any vendor who quotes you Rakuten 24's 53 percent as your expected return is misreading a case study, or hoping you will.

The position worth holding is a measured one: Core Web Vitals can move revenue, the size of the move is specific to your site and can only be found by measuring it, and the work that pays off is fixing the faults on your own page against the thresholds Google reads, not chasing someone else's percentage.