Platforms as Governments: Content Moderation and the Oversight Board

A revenue model that outran its safety spending

Myanmar opened to smartphones and mobile data quickly after decades of military-era isolation, and Facebook arrived at the center of that opening. Zero-cost data arrangements and cheap Android handsets meant that for a large share of the population, the Facebook app was not one service among many; it was the practical entry point to the internet itself, the place people read news, ran small businesses, and organized community life. A platform in that position carries the reach of a national utility, whether or not it is built or staffed like one.

Facebook's own staffing record shows it was not built like one, at least not in Myanmar's language. The company had one person monitoring Burmese-language content in 2014, hired a second in early 2015, and had reached roughly 60 Burmese speakers by mid-2018, according to Reuters reporting later cited by CBS News. A Reuters and UC Berkeley Human Rights Center investigation published that August found more than 1,000 posts calling for violence or genocide against the Rohingya still live on the platform, some of them for at least six years, despite a Facebook operation that was supposed to catch exactly this content.

The gap is an economic one before it is anything else. An advertising business built to operate across more than a hundred languages extracts value from every market it reaches, largely in proportion to how many people use it and how long they stay. Safety staffing does not scale the same way. It has to be hired language by language, market by market, and a country that is highly engaged but not yet a large advertising market can go years without the moderation headcount its user base would justify if judged by reach alone. Myanmar, with over 50 million people and a Facebook-dependent information environment, sat on the wrong side of that mismatch for the better part of a decade.

Facebook eventually admitted as much in public terms, stating it had been too slow to act on hate speech in Myanmar and that it was working to prevent the further spread of misinformation and hate, a statement reported by Hindustan Times in August 2018. The admission came only after outside reporting made the staffing gap and the surviving posts visible to the public, which is itself a data point: a company that can quietly under-resource a market for years is not one whose own internal review can be assumed sufficient on its own.

What investigators found, and what it drove

The consequences arrived in 2017, when a military crackdown in Rakhine State drove a mass exodus of Rohingya out of Myanmar. More than 700,000 people had fled to neighboring Bangladesh since October of that year, one of the fastest-moving refugee crises of the decade. NBC News reported that UN investigators, examining the role of information and incitement in the violence, singled out Facebook specifically.

The investigators' language was unusually direct for a body normally cautious about attributing cause. They concluded that disinformation campaigns hosted on the platform played a determining role in provoking the violence against the Rohingya. That is a stronger finding than "contributing to" or "amplifying"; it assigns Facebook-hosted content real causal weight in an atrocity, a judgment more commonly reserved for state propaganda organs or wartime broadcasters than for a private social network.

The legal reckoning followed years later. In December 2021, Rohingya refugees filed a lawsuit seeking $150 billion in damages from Facebook, alleging that the platform's algorithms amplified the hate speech that fueled the 2017 violence, reported by Al Jazeera at the time of filing. Whatever the suit's eventual outcome, its scale (a figure comparable to the market value of major public companies) signaled how far the claimed harm had traveled from a normal content-moderation dispute.

The geopolitical thread here is the plainer of the two. A private company's ranking and moderation choices, made in offices thousands of miles from Rakhine State, were found by international investigators to have carried consequences on the scale states usually reserve for their own conduct: displacement of hundreds of thousands of people, and a genocide inquiry. No treaty, licensing regime, or international body had been built to hold a communications platform to that standard before the fact. The Myanmar case forced platforms worldwide, and the governments that regulate them, to accept that a content-ranking decision is not a neutral technical choice; it is an act with the kind of reach and consequence that used to belong only to states.

From apology to institution: building a private Supreme Court

Facebook's public response, beyond the apology, was structural. In November 2018, Mark Zuckerberg approved the creation of an independent Oversight Board after a meeting with Harvard Law professor Noah Feldman, describing the planned body as a kind of Supreme Court for the platform, one with the power to override the company's own content decisions, according to the board's own documented history on Wikipedia.

No prior communications company, whether a newspaper, a broadcast network, or an earlier generation of internet platform, had built a body with binding authority to overrule its own leadership on individual pieces of content, reviewable case by case. Myanmar was not the only pressure driving the decision; 2016 election interference and the Cambridge Analytica data scandal had already put Facebook's judgment under sustained public scrutiny. But Myanmar was the case that made the humanitarian stakes of a single company's unilateral content calls impossible to argue away, and it became the reference point for why an external check was needed rather than another round of internal policy revision.

The construction carries an obvious tension the Myanmar case exposed in the first place: a board created, funded, and staffed by the company it reviews is not the same thing as a judiciary built by constitutional design, answerable to an electorate or a legislature outside the institution it checks. Facebook's framing borrowed the language of separated powers without the structure that normally enforces it. Whether that borrowed authority would function as more than a public-relations instrument was a question the board's early record would have to answer, not its charter.

The board at work

The Oversight Board moved from announcement to operation over roughly two years. Its 20 founding members were announced in May 2020, and the board formally began its work on October 22, 2020, according to its documented history.

Its first public test came on January 28, 2021, when it issued its first five decisions and overturned Facebook's original action in four of the five cases. For a body created by the company whose decisions it was reviewing, a four-out-of-five reversal rate in its opening act was a meaningful signal: it suggested a willingness to bind Facebook in practice, not simply to ratify choices the company had already made.

The board took on its highest-profile case soon after: a review of Facebook's decision to suspend then-president Donald Trump following the January 6, 2021, attack on the US Capitol.

The law that lets platforms decide

None of this happened in a legal vacuum. In the United States, platform content-moderation liability runs through Section 230 of the Communications Decency Act, a provision that shields platforms from liability for content their users post while separately protecting their right to moderate that content as they see fit. Section 230 is the reason a body like the Oversight Board could exist as Facebook's voluntary internal check rather than a legally mandated one; American law treats the moderation choice itself as the platform's private editorial decision.

That settlement has been fraying. The scope of Section 230 and of platforms' moderation authority has reached the US Supreme Court in cases including Moody v. NetChoice, LLC, part of a wider contest over whether individual states can pass laws compelling or restricting how platforms moderate content on their own services, per documented case history on Wikipedia. The legal question in the United States is narrower than the one Myanmar raised: not whether platforms wield governing-scale power over speech, which both sides of that fight now take as given, but who inside the American constitutional system, the platform, the states, or Congress, gets to set the rules for how that power is used.

Outside the United States, the fight has a different shape. There is no equivalent domestic legal fight over who may compel a platform's moderation choices in most of the markets where platforms operate, including Myanmar in 2017. What exists instead is the plain fact the Rohingya case established: a platform can carry governing-scale influence over what a population sees and believes, without the accountability structures, elections, courts, or constitutions, that bind an actual government to answer for the consequences.

The unresolved question, from feeds to AI answers

The Oversight Board's record is a real, if partial, answer to the accountability gap Myanmar exposed. It is the first instance of a platform building a body with public, binding authority to overrule its own executives on individual content decisions, and its early record, reversing four of its first five cases and taking on the review of a former head of state's account, shows it was not designed purely for appearance. But it addresses one company's feed, reviewed case by case, years after the crisis that prompted it. The deeper question it was built to answer, who checks a private company's power over what a population is allowed to see, was never fully resolved; it was only given, for the first time, a formal venue.

That question has since moved to a newer surface. Where feeds once decided what a person saw, AI answer engines increasingly decide which businesses get named, cited, and recommended when a person asks a question rather than scrolls a timeline. The mechanism is the same one the Myanmar case exposed: a private company's ranking choices determine what reaches an audience, at a scale individual businesses and, in aggregate, whole markets cannot see or contest from outside. No body resembling an Oversight Board yet exists for the systems making those decisions, and it is an open question, not a settled one, whether the pressure that produced Facebook's board will produce anything comparable for the AI layer now sitting between businesses and the people trying to find them.

What the Myanmar case leaves as its clearest lesson is not a verdict on any one company's intentions. It is a measurement problem stated in the plainest terms available: a platform's safety spending in a market can lag years behind the revenue and reach it draws from that market, and when it does, the cost of the gap is not paid by the platform. Facebook's Oversight Board was one company's attempt to close that gap after the fact, in public, and under its own name. Whether the newer systems making similar decisions today will be measured, and held to standards, before their own version of that gap becomes visible is the question the last decade of platform governance leaves open.