The AI Productivity Case: What a Peer-Reviewed Study of 5,000 Support Agents Actually Shows

The study, and why it carries more weight than most

Most claims about AI and productivity rest on vendor case studies, self-reported surveys, or short demonstrations. Brynjolfsson, Li and Raymond published something different: a field study of the staggered rollout of a generative-AI conversational assistant to 5,179 customer-support agents at a large software firm, released as NBER Working Paper 31161 in 2023 and later peer-reviewed and published in the Quarterly Journal of Economics in 2025.

Two features give it unusual authority. First, the outcome measured was a hard operational number, issues resolved per hour, drawn from the firm's own logs rather than from what workers said about themselves. Second, because the assistant was introduced to different groups at different times rather than all at once, the researchers could compare agents who had the tool against otherwise similar agents who did not yet have it, which is far closer to a controlled comparison than a simple before-and-after. That design is why this study, rather than the louder claims around it, is the reference point for what generative AI does to knowledge work.

The headline number, and the distribution beneath it

On average, access to the assistant raised productivity, measured as issues resolved per hour, by about 14 to 15 percent. That figure is worth stating plainly because it is neither trivial nor miraculous. It is a meaningful operational improvement of the kind that compounds over a year, and it is a long way from the wholesale replacement of labor that both enthusiasts and alarmists tend to describe.

The more instructive result is how unevenly that average was distributed across the workforce.

The mechanism: generative AI at work spread tacit knowledge

The authors' explanation for the pattern is precise and testable. The assistant, trained on the firm's own record of successful resolutions, effectively captured the tacit best practices of its most able agents and made them available to everyone else in the moment of a customer conversation. It functioned less like a machine doing the work and more like an always-available senior colleague suggesting how a strong agent would handle the case.

That mechanism explains the distribution. If the value of the tool is the diffusion of expert practice, then the people with the most to gain are precisely those furthest from that practice, and the people with the least to gain are the experts who supplied it. It also reframes what such a system is for. Its economic contribution in this study was not to remove humans from the loop but to raise the productivity of the humans who were least productive, narrowing the spread between the best and the rest.

Beyond throughput: sentiment and retention moved too

Productivity was not the only outcome that shifted. The same study reported that the assistant improved customer sentiment, measured in the tone of interactions, and was associated with higher worker retention. Both effects are consistent with the mechanism. Customers reach resolution faster and with fewer poor exchanges when a struggling agent has better guidance, and agents who feel supported rather than exposed in difficult conversations are more likely to stay.

For any operator weighing an AI deployment, these secondary results matter as much as the headline. A change that speeds up work while degrading the customer experience or burning out staff is not a durable gain. Here the evidence pointed the other way, which is part of why the study is cited as a case for augmentation rather than mere speed.

Augmentation vs automation: what the wider evidence says

The support-agent study is one data point in a larger and more cautionary picture, and the accurate reading places it inside that context rather than treating it as a promise.

Anthropic's Economic Index, built from usage telemetry across its business customers, found that when firms embed the model through an API, roughly 77 percent of transcripts show full-task-delegation patterns and only about 12 percent show iterative, collaborative augmentation, the reverse of the ratio seen in consumer chat use. In other words, businesses tend to reach for automation by default even though the best-documented workplace gains in the support study came from augmentation. That gap between how AI is deployed and where its measured value has actually appeared is the practical risk.

Two further lines of evidence explain why default automation of a customer-facing role is fragile. In the peer-reviewed tau-bench benchmark, frontier LLM agents succeeded on fewer than half of realistic customer-service tasks, and consistency was worse still: asked to repeat the identical task eight times, the same agent succeeded only about a quarter of the time. And Klarna, after publicly replacing large parts of its support workforce with an AI assistant in 2024, resumed hiring human agents by 2025 and moved to a hybrid model, with its leadership stating that customers should retain the option to reach a person. The throughput gain in the support study is real; the reliability required to remove the human entirely is not yet demonstrated in the same rigorous way.

The limits: one large firm is not your front desk

The result should be held with clear caveats.

The study observed a single large software firm's contact center with thousands of agents handling a high, repetitive volume of text-based issues. That is close to an ideal setting for the mechanism to work: enough historical data to learn from, standardized tasks, and a large enough novice population for the floor-raising effect to show up in the aggregate. A five-person med-spa front desk or a solo-legal intake process shares some of these features and lacks others, and no study surfaced here replicates the finding at small-business scale.

Extending an enterprise contact-center result to a small team is therefore a reasonable inference, not a proven transfer, and it should be labeled as such. The 14 to 15 percent figure is a finding about one workforce, not a number any business should expect to reproduce. What travels more safely than the specific percentage is the shape of the result: that generative AI in this setting helped least-experienced workers most, worked by diffusing expert practice, and delivered its value alongside humans rather than instead of them. That shape is a hypothesis a small business can test on its own workflows, with its own baseline, rather than a promise to be taken on faith.

What a small business should take from this

The operational reading of the evidence is narrower and more useful than the headline. It suggests aiming an AI deployment at the parts of the work where the least experienced people lose the most time, drafting and researching and structuring a response, while keeping a human on the decisions that carry risk. It suggests measuring a genuine before-and-after on your own numbers rather than importing someone else's percentage. And it suggests treating full, unattended automation of a customer-facing role as an unproven step to be earned through evaluation, not assumed at the outset, given the benchmark and real-world reliability evidence above.

None of this argues against using AI. It argues for a specific way of using it: as a multiplier on a team, weighted toward its newer members, with the human kept in the loop where it matters and the effect measured rather than asserted. That is a defensible position drawn from the strongest evidence available, and it happens to be the opposite of the reflex the wider deployment data shows most businesses following.