The Yelp Effect, Ten Years Later: What a Regression Discontinuity Actually Proved

What the Yelp effect actually is

In a 2011 Harvard Business School working paper, "Reviews, Reputation, and Revenue: The Case of Yelp.com," Michael Luca set out to answer a question that sounds simple and is not: does a better Yelp rating cause a business to earn more, or do good businesses simply earn both better ratings and more revenue for the same underlying reason?

The headline result is precise. A one-star increase in a restaurant's Yelp rating is associated with a 5 to 9 percent increase in revenue. That is the figure that entered the industry's vocabulary as "the Yelp effect." But the figure matters far less than the design behind it, because the design is what lets Luca say cause rather than correlation, and it is the design that also fences off exactly where the claim stops being true.

Why a regression discontinuity can claim cause, not just correlation

The obvious way to study ratings and revenue is to line up businesses by their star rating and see whether higher-rated ones earn more. They do, but that comparison proves nothing about cause. A restaurant with a genuinely better kitchen will tend to earn both a higher rating and higher revenue, so the two move together without one driving the other. This is the endogeneity problem, and it defeats almost every casual claim about reviews.

Luca's move was to exploit a quirk in how Yelp displays ratings. The site shows a rounded star rating in half-star increments, even though it computes the underlying average to far more decimal places. A restaurant with a true average of 3.24 and one with a true average of 3.26 are, on quality, essentially identical, yet the site rounds one down to 3.0 stars and the other up to 3.5. The displayed rating jumps at the rounding threshold while the actual quality does not.

That discontinuity is the natural experiment. Businesses sitting just on either side of a rounding cutoff are, on average, interchangeable in every respect except the star rating a buyer sees. Comparing revenue across that narrow boundary isolates the effect of the displayed rating itself, cleanly separated from underlying quality. To measure revenue, Luca matched the ratings data to Washington State Department of Revenue tax records rather than relying on self-reported or estimated figures. That is what earns the study its authority: a credible identification strategy joined to hard administrative revenue data.

Does a Yelp rating affect sales, and only where the study looked

The most important sentence in the paper is not the 5 to 9 percent. It is the finding that the effect is driven entirely by independent restaurants. Chain restaurants show no meaningful relationship between their Yelp rating and their revenue at all.

The reason is intuitive once stated. Buyers already hold strong prior beliefs about a chain. A person choosing a national franchise knows roughly what they will get before they read a single review, so the marginal star moves nothing. An independent restaurant has no such prior working for it. For the independent, the Yelp rating is often the only credible third-party signal a buyer has, so it carries the full weight of the decision. The revenue effect lives exactly where the information gap is widest.

Luca also documented that this dynamic is associated with a measurable decline in chain market share over the period studied, consistent with review platforms shifting attention toward independents that were previously hard to evaluate. That is a genuine and underappreciated part of the finding: review sites did not just reward good ratings, they partially re-leveled the field between independents and chains.

What the study does not prove

Because the number is quotable, it gets stretched far past what the evidence supports. Three limits deserve to be stated plainly.

The older correlational evidence agrees on direction, and on asymmetry

Luca's work did not appear in a vacuum. Five years earlier, Chevalier and Mayzlin compared relative book sales across Amazon.com and BarnesAndNoble.com and found that a one-star improvement in a book's average rating correlated with up to a 9.9 percent increase in relative sales. That study is correlational rather than causally identified, so it carries less weight on the question of cause, but its direction and rough magnitude are consistent with Luca's later, cleaner estimate.

Chevalier and Mayzlin added a finding that reframes how a business should think about its reputation: the impact of one-star reviews was larger in magnitude than the impact of five-star reviews. Negative information moved buyers more than an equivalent amount of positive information, an asymmetry consistent with loss aversion. Put together with Luca, the practical reading is that a bad rating is not the mirror image of a good one. The downside of a poor reputation is heavier than the upside of a strong one, which is an argument for defense, not just accumulation.

How the effect gets misread ten years on

The gap between what the study proved and how it is cited has a predictable shape. The common misreadings are worth naming directly.

• Treating "5 to 9 percent per star" as a universal constant. It was estimated for independent restaurants in Washington State. Applying the exact number to a med spa, a law firm, or a home-services contractor is an extrapolation, not a finding.
• Assuming the effect exists for every business type. It did not appear for chains at all, because buyers already held strong priors about them. Any business with a strong pre-existing brand should expect a smaller marginal effect than an unknown independent.
• Confusing this causal study with the many correlational ones. Most "reviews drive revenue" claims online rest on studies that cannot separate cause from quality. Luca's can. That distinction is the whole reason the paper is cited, and collapsing it flatters weaker evidence.
• Reading a persuasion effect as a quality guarantee. The rating moves buyers whether or not it accurately reflects quality, so a rise in stars is not proof the underlying service improved.
• Assuming the 2011 restaurant result transfers unchanged to today's AI-answer surfaces. That is a live extrapolation, examined below, not something the study established.

Does the effect transfer to the AI-answer era?

It is tempting to assume that if a star rating moved restaurant revenue on a list of Yelp results, it must move revenue even more now that an AI engine reads reviews and names a single business in its answer. The intuition is reasonable, and the underlying mechanics, a trusted third-party signal filling an information gap, plausibly still apply. But it should be labeled for what it is: an extrapolation, not a measured result.

The causal literature, Luca included, studied ranked lists and human browsing. It did not study single-answer generative surfaces, which did not exist when the data was collected. When an engine synthesizes one recommendation instead of returning ten, the same trust and social-proof mechanics may concentrate onto whichever business the engine names, raising the stakes of the rating rather than lowering them. That is a defensible hypothesis and a reason to take reputation seriously in the AI-answer era. It is not something the regression discontinuity proved, and anyone who claims it did is overreaching.

Reputation is now a regulated, adversarial system

One thing has changed unambiguously since 2011. Reputation is no longer an honor system. In October 2024 the Federal Trade Commission's Trade Regulation Rule on the Use of Consumer Reviews and Testimonials, 16 CFR Part 465, made fake, incentivized, insider, and suppressed reviews federal violations carrying civil penalties of up to 51,744 dollars each. At the same time, Google's own Search Quality Rater Guidelines name trust as the load-bearing member of its Experience, Expertise, Authoritativeness and Trust framework, the criterion the other three feed into.

Read alongside the academic record, the message is consistent. A rating genuinely moves revenue, especially for an independent with no brand to fall back on. That rating is a persuasion signal more than a quality one, it responds asymmetrically to bad news, and it now sits inside a compliance regime where the shortcuts are illegal. Building a reputation that earns the Yelp effect honestly, and can survive scrutiny, is a discipline, not a growth hack.