I get asked a lot whether AI has solved localization.
It’s almost a redundant question. Some content can now go out without anyone checking it. But translation as a whole is still an unsolved problem.
The hard part is convincing anyone, because of what I call the blind effect.
You run a product page through a large language model, the Japanese comes back in seconds, it looks like Japanese, and off it goes. Usually that’s fine. When it isn’t, a native speaker would spot the problem straight away, but the person pressing publish usually isn’t one. I do the same with AI-written code I can’t fully read: it runs, so I trust it. With language, if it renders, it ships.
That’s why human judgment in AI localization matters more as the technology improves: the better the output reads, the easier its mistakes are to miss.
I co-created COMET, which much of the field now uses to score machine translation automatically. I’m a fan of automation. I also know where it stops.
Why fluent AI translation can still be wrong
Large language models do astonishing things, mostly in English. Quality is strongest in the languages that dominate the internet and falls away beyond them. WMT, the field’s main annual evaluation, still describes translation into low-resource languages as a continuing challenge.
Its 2025 findings include a near-perfect example of a ‘silent failure’. Systems asked to translate into Maasai often produced Swahili, and systems asked for Egyptian Arabic tended to produce Modern Standard Arabic. A speaker of either would notice at once. Automated quality checks probably wouldn’t, the organizers note, since they rarely check which language they’re reading.
Scoring tools won’t close that gap on their own, and I say that as someone who builds them. The same year’s research on evaluation metrics found they can still be fooled by fluent output that misses the meaning, and that they track human judgment poorly in low-resource languages. They’re weakest exactly where you can least check the output yourself.
“The better the output reads, the easier its mistakes are to miss.”
The failures that aren’t about language at all
Some of the most expensive failures have nothing to do with grammar.
Take terminology. The approved name for a product, or a phrase your legal team spent weeks agreeing on, comes back three different ways in three markets. Each reads beautifully. None of them is yours.
Or take the HTML and XML tags around web text, the invisible markup that tells a page what’s a link and where a button goes. Tags aren’t natural language, and the deeper the markup, the more likely a model drops or misplaces one, in any language, English included. I’m working on exactly this right now, and it’s unforgiving: you can’t build a website if the tags are broken.
A broken build at least announces itself. A wrong product name doesn’t, and teams that switch AI on and walk away tend to find those one incident at a time.
Where human judgment sits in AI localization now
So yes, we still need human judgment.
It’s just doing different work, in four places…
Deciding where the risk is
The most important call now happens before anything is translated: what would this cost us if it were wrong, and who should check it?
For years, every sentence was held to the same standard, sometimes so strictly that a punctuation question could hold up a release. Some content deserves that rigor. Applied to everything, it’s a bottleneck.
Legal, medical and other regulated content is always high risk and needs expert human review. Product and website copy sits in the middle. Much user-generated content needs light review, or none. The thinner a language’s resources, the further up that ladder its content should move. Deciding where each piece sits is increasingly a leadership call: in independent research Phrase commissioned this year, 47% of 550 senior business leaders said they were moving too fast on AI without governance.
Building expertise into the system
Some of the best judgment in a workflow never reads a page. It’s built into the system by people who know what goes wrong. The model gets your translation memories, terminology and context, and checks hunt for the errors fluent output hides. That’s been the focus of our research at Phrase for the past two years, across many customers, and the approach changes with every new model.
One animal-health company we work with built a simple check comparing each translated string with the label it’s filed under. It caught “lymphoma,” a cancer, rendered as “lipoma,” a benign fatty lump, after a full human review cycle. The reviewers weren’t careless. They were human. The judgment that caught it was human too. It just lived in a system someone designed.
“The reviewers weren’t careless. They were human.”
Steering by what happens in market
Even good systems need steering, far more than people expect; it still surprises me how much I do in my own work. Left alone, a model drifts and fails at simple things nobody catches.
In localization, steering looks like this. You launch in a new market, and satisfaction and engagement come in below expectations. Someone reads that signal, works out why, and adjusts the system. That’s where a localization specialist starts to look like a strategist. It’s also why, as I’ve argued before, market signals tell you more than an accuracy score.
Reading what the text can’t convey
A model sees the text you give it, in that moment. It may well know more about business etiquette in Japan than anyone on your team. What it can’t always tell is when that knowledge applies.
Picture a nurture email that opens “Hi {first_name}.” The model renders the greeting in flawless Japanese and leaves the placeholder alone, as it should. The email still misfires, because Japanese business correspondence typically uses the family name with an honorific. The model knows the convention. But nobody asked it to look at the template, which is where the mistake lives.
Multiply that by sales conventions and what counts as pushy in each market. The text is fine; we’re applying it in the wrong way.
Sometimes that falls flat.
And sometimes it does real harm.

What this means for the people doing the work
The shift is real, and it isn’t tidy. Some early adopters cut localization teams, assuming the work was automated. In high-risk fields like medicine and law, many teams work much as they always have, often for good reason. Most organizations sit in between, working out where language expertise belongs.
What I see most is movement outward. Specialists who once sat in a central team that content queued behind are joining content, product and engineering teams, next to the people who create the content. Their question changes from “can you translate this?” to “what should we say in this market so it feels made for the person reading it?”
Giulia Greco, Director of Localization at the Project Management Institute and formerly Shopify’s Head of Localization, has built a keynote around how localization teams earn that kind of seat across their companies. Her advice is clear:

It’s now rare to meet a localization professional who hasn’t tried these tools. Some fear remains, and some teams have simply been told to adopt AI. But the most capable people I know have leaned in, and their expertise is what makes the technology work.
A localization lead at a leading hospitality technology brand, working under an AI-first mandate, told us how the work has changed. The team used to spend much of its time briefing, routing and chasing work. Now more of it goes on deciding what needs expert review, setting up the systems that handle routine work, and checking results in market.
Who owns the words
Whatever the model does, someone still owns what your company says in 30 languages. AI will take on more of the work every year. The accountability stays with us.
So the useful question for any leader is where judgment sits in your workflow today, and whether it’s built in or simply hoped for.
About the author: Craig Stewart is Director of AI Research at Phrase. A Carnegie Mellon graduate with a background in translation and linguistics, he co-created COMET, an industry-standard metric for machine translation quality.





