The Script Problem: Devanagari, Roman Hindi, and the MSME That Doesn't Exist Twice

One business, two names, and neither is wrong

Hindi has two scripts in everyday use. Devanagari is the constitutionally recognized script, the one taught in school and printed on a signboard. Romanized Hindi, the same words spelled out in Latin letters, is what most people actually type, because it is the script native to the keyboard on their phone. A sweet shop's board might read मिठाई की दुकान; the same owner's WhatsApp Business profile might read "Mithai ki Dukaan." Both are correct. Neither is a typo of the other. They are two legitimate renderings of one name.

For a human reader that duality is invisible friction, a fraction-of-a-second switch most Indians make without noticing. For a machine that reads text as strings and tokens, it is a different fact pattern. Two spellings of a name are, absent extra work, two different strings, and any system built to decide who is who has to actively resolve them as the same entity rather than assume it. Nothing in how the modern web is crawled guarantees that resolution happens.

This is not a claim that every MSME with a script mismatch has been proven invisible somewhere. It is a claim that the mechanism for that kind of invisibility is documented, current, and worth understanding on its own terms, separate from the broader vernacular-search story.

Why a string match is not an entity match

Large language models and search systems do not read a name the way a person does. They break it into tokens, and how a script fragments into tokens changes what the model does with it. A 2026 study measuring tokenization cost across fourteen languages found that under the tokenizer used by GPT-3.5 and GPT-4, Indian languages face an average 8.0x tokenization cost relative to English, rising to 13.0x for Malayalam, which shrinks the effective context window available for the same content to as little as 12% of what an English speaker gets. Native-script Devanagari text is expensive to process in exactly this sense.

Romanizing the same text changes the arithmetic, because it increases lexical overlap with the English-heavy vocabulary most general-purpose tokenizers were built on. Several recent studies on romanized Hindi, Bengali, and Nepali describe this as a "tokenization dead zone": romanized Indic text is neither standard English nor native-script text, so it gets processed differently from both, with its own fragmentation problems. In practical terms, a business name in Devanagari and the same name transliterated into Roman letters do not just look different to a person switching scripts; they travel through different token paths and land in different neighborhoods of a model's internal representation. That is the technical seam beneath the "two identities" problem.

Google already found this gap in its own map

The clearest evidence that this is a real, current problem, not a theoretical one, comes from Google itself. Google's Maps research team has documented that most Indian points of interest, restaurants, shops, hospitals, bus stops, were named online only in English or Latin-script text, often mixed with local words, even in markets where the primary reading script is Devanagari or another Indian script. Google built an ensemble of machine-learning models to automatically generate native-script transliterations for these listings, and reported that the effort increased native-script name coverage by as much as twenty-fold in some Indian languages.

That is a company running one of the largest local-business indexes in the world, describing a script-coverage gap large enough to justify a dedicated research and engineering program. It is a reasonable inference, though not a proven certainty for every business, that comparable gaps exist wherever a business's only online presence lives in a single script, and that an AI answer engine drawing on that same web has no more reason than pre-fix Google Maps did to assume the Devanagari and Romanized versions of a name refer to one place.

Separately, Google built MuRIL, a multilingual language model, specifically because earlier general-purpose multilingual models underperformed on transliterated and Romanized Indic text. Its training deliberately pairs monolingual text with translated and transliterated versions of the same content, an engineering choice that only makes sense if the two forms were not being handled as equivalent by default.

A demand mismatch layered on a supply mismatch

The audience on the other side of this problem is not a niche. A 2017 study by KPMG and Google projected that Indian-language internet users would reach 536 million by 2021, about 75% of India's total internet base, against 199 million English users, and found that Indian-language users were far more likely to respond to advertising in their own language than in English. The vernacular audience was already, on that projection, the majority audience for the Indian internet.

Yet the reference material an AI system can draw on to corroborate a business named in Devanagari is comparatively thin. As of August 2026, Hindi Wikipedia has 170,753 articles against roughly seven million on English Wikipedia, and more than 85% of Wikipedia pageviews originating from India still go to the English edition. A Devanagari-named business is competing for corroboration inside a noticeably smaller reference corpus than an English or Romanized one, even though the readers who would search for it in Devanagari outnumber, by a wide margin, the readers searching in English.

Put the two facts together and the shape of the problem sharpens: a large and growing vernacular audience, sitting on top of a thin vernacular reference corpus, with the same business potentially split across two scripts that a machine has no guaranteed way of recognizing as one name.

The tools exist; systematic coverage does not follow automatically

None of the individual technical pieces are missing. A formal international standard for transliterating Devanagari and related Indic scripts into Latin characters, ISO 15919, has existed since 2001, so the character-mapping problem itself was solved decades before generative answer engines existed. Schema.org's alternateName property lets a business declare a secondary name, in principle including a transliterated one, inside its own structured data. India's government-run Bhashini mission, launched in July 2022 under the Ministry of Electronics and Information Technology, offers open translation, transcription, and transliteration APIs across the 22 scheduled languages as public digital infrastructure, free for developers to build on.

What none of these solve by themselves is the systemic, entity-resolution step: given a Devanagari name found on one page and a Romanized name found on another, deciding, reliably and at web scale, that they are the same business. That is precisely the gap that ParaNames 1.0, a 2024 academic project, was built to address. Its authors constructed a 140-million-name corpus spanning more than 400 languages and 16.8 million Wikidata entities specifically because reliable cross-script and cross-lingual name matching was not a solved problem even for Wikidata, arguably the best-corroborated, most heavily curated entity dataset on the open web. If cross-script matching still needed dedicated research infrastructure at that scale, an unlisted MSME with a Devanagari Facebook page and a separately spelled Romanized Google listing has no structural reason to assume any crawler resolves the two on its own.

What has, and has not, actually been measured

It is worth being precise about where the evidence stops. No public, audited study was located during this research that directly tests whether a named generative answer engine, ChatGPT, Perplexity, Google AI Overviews, or Gemini, treats a real business's Devanagari-only listing and its Romanized-only listing as one entity or two, and reports the resulting difference in whether, and how, the business is surfaced. What exists is indirect but consistent: documented token-cost asymmetry between native-script and Romanized Indic text, academic evidence that cross-script entity linking remains an open research problem even at Wikidata's scale, and Google's own documented admission, through its Maps research, that most Indian business listings existed in only one script until the company built a dedicated system to fix it.

A reproducible test of the specific claim would need matched pairs of real businesses, each queried under both its Devanagari and Romanized name across a panel of answer engines, scored on whether the engine surfaces the business at all and whether it returns consistent details, repeated across multiple phrasings and time points to control for the normal volatility of generated answers. To our knowledge, that dataset does not yet exist publicly for Indian MSMEs specifically. Until it does, the script-duality visibility gap is a well-evidenced open question, not a measured fact.

Applying this to an individual business

The mechanism is documented at every step except the last one: whether it currently costs a specific business visibility inside a specific AI answer engine has not yet been measured and published for Indian MSMEs. That gap between a documented mechanism and a measured outcome is exactly where a diagnostic belongs, rather than a claim in either direction.

The practical takeaway does not depend on resolving the open question first. Where a business's name, address, or offering lives in only one script anywhere it is listed online, whether that is Devanagari on a signboard photo and nowhere else, or a Romanized name on a delivery app and nothing in Devanagari, that business has the same condition Google's own Maps team found large enough to fix at national scale. Closing that condition, consistently, everywhere the business is named, is a defensible action regardless of how the still-open measurement question eventually resolves.