When revenue pressure overrides AI governance

What happens when a governance leader discovers that an unaudited AI model has become critical to revenue?

At a recent Responsible AI Conference, leaders from major AI-driven companies were asked that question. More than 70% of the people in the room said they would allow the models to continue operating while they worked out how to bring them under existing governance processes.

The result came from an informal conference poll and serves as an indication of the pressure governance leaders face once an unapproved system becomes commercially valuable.

Governance is easier to enforce while an AI system is still being tested. Once it becomes part of customer interactions or core workflows, suspending it can put revenue at risk.

Dr. Eva-Marie Muller-Stuler, Founder and Chief AI Officer at The Hummingbird Group, has held senior AI and data leadership roles at IBM and EY. She has also built governance frameworks for the Middle East’s financial sector.

She witnessed the exchange firsthand. Speaking on Phrase’s In Other Words podcast, she was unequivocal.

“Guys, that is not your job. Your job is to stop it. Because if your plane runs on something that is not audited, it might crash at any second. If your AI model is signing off credits, you’ll find out five years later if they’re wrong or right.”

Her concern is that commercial dependence can make corporate controls increasingly difficult to enforce.

How exceptions become operating models

An unaudited model rarely becomes revenue-critical overnight. It may begin as a pilot designed to test a narrow use case. Early results attract internal support, and more teams begin relying on it. The system gradually becomes part of the operating environment.

Formal approval is deferred because the model is still described as temporary. The business keeps using it while governance work is scheduled for later. By the time leaders recognize the exposure, the system is embedded in the workflow and tied to revenue.

This creates a strong incentive to grandfather it into the governance framework. The organization preserves continuity while promising to address the controls retrospectively.

Over time, an exception becomes accepted practice. The burden moves from proving that the system was ready for deployment to proving why it should be suspended.

Governance standards can weaken through a series of reasonable short-term decisions. The organization may still have a responsible AI policy and a committee that reviews new use cases. Its most commercially important systems can remain outside the controls those structures were created to enforce.

Governance requires authority

Georg Ell, CEO of Phrase, addressed a related accountability problem in a recent Forbes article.

“In most enterprises today, no single person is accountable for what AI communicates to customers globally,” he wrote. “The responsibility is distributed across marketing, product, legal and technology teams.”

The conference response raises a further question. Even when someone is accountable, do they have the authority to stop a system that is generating revenue?

A governance leader may set standards and document risk. Suspension authority may sit elsewhere. Decision rights often remain with the executive who owns the revenue or the product. Governance then becomes advisory at the moment when enforcement matters most.

A company may have named owners for responsible AI while leaving them without a mandate to intervene.

Research from the RAND Corporation has examined why many enterprise AI programs fail to progress from pilots to sustained value. Organizational readiness is a recurring factor, particularly when decision rights and operating structures remain unclear.

Dr. Eva has a name for the organizational behavior that fills this void. 

“The CEO comes back from a conference, they sit in the board meeting, and they’re like, oh, we really need to do something with AI. And then they spin off what they normally do. Let’s create a roadmap. Let’s collect some use cases. Let’s talk to the vendors, build pilots, and then celebrate the demos. And that’s where they normally end.” 

Her assessment is blunt.

“This is not really progress. This is motion.”

The number of pilots in production says little about whether an organization can control them once they become commercially important.

The consequences are harder to see across markets

Customer-facing AI creates an added layer of exposure because the consequences of weak governance may emerge gradually. The greatest exposure often sits in markets that receive the least executive scrutiny.

A model can generate fluent communication while failing to account for the customer receiving it. Language and cultural context shape whether a message is credible.

Large language models reflect the sources and populations most visible in their training data. Their outputs can carry assumptions that work for one audience and become less appropriate elsewhere.

Dr. Eva, who is based in Dubai and has worked extensively across the Gulf region, sees this directly.

“All the content we are producing is heavily dominated by white, rich, democratic demographics. The advice you would get is always the advice that a white male American should get.”

She points to audiences who may be poorly represented in the underlying data.

“It might not work for an Emirati woman. I say Emirati because I live in Dubai, women over 65, because they’re not represented in that data set at all.”

She also cites an incident involving Germany’s rail system. An app interpreted “Std.,” the German abbreviation for hours, as “STD”, a reference to a sexually transmitted disease. The error became a humorous news story, but it shows how easily an automated system can misinterpret meaning.

The same error in a medical setting could turn an instruction to return in one hour into something resembling a diagnosis.

Georg described the broader pattern in Forbes. A company’s AI content program may perform well in North America, while engagement declines in other markets because the messaging misses cultural context and gradually undermines credibility.

“Small misses can accumulate quietly until customers start to disengage and they rarely tell you why.”

The Global Content Disconnect report, based on independent research among 550 senior leaders across nine countries, found that 96% had made AI central to their content strategy. At the same time, 47% acknowledged that their organizations were moving too quickly without adequate governance.

AI-generated communication can become commercially valuable before an organization has established how quality will be assessed across markets.

Once the system is tied to growth, leaders may become more willing to accept weaknesses that would have prevented approval at an earlier stage.

Governance must withstand commercial pressure

Effective governance starts before deployment. The operating model needs clear approval requirements and defined suspension authority. Escalation rules should also show how challenges from commercial owners will be resolved.

Mazak, the Japanese machine tool manufacturer, encountered a related challenge when it relaunched its global web presence.

The company was managing 29 regional websites in 20 languages through fragmented manual workflows. Different regions were using inconsistent processes, creating uneven customer experiences.

Mazak redesigned the workflow so that content creation and translation moved through a common platform. Shared language assets and automated quality checks were applied across regions.

Mazak’s experience shows why controls are easier to enforce when they are embedded before a workflow becomes commercially essential.

The same principle applies to AI agents.

They need access to the purpose of the communication and the audience receiving it. They also need the approved standards that determine whether an output is suitable for use.

Semih Altinay, VP of AI Solutions at Phrase, has addressed this at Loc360° Tokyo and SlatorCon Silicon Valley 2025.

Semih has argued that AI agents need access to intent and audience context before they generate content. That context gives organizations a basis for evaluating whether the output is appropriate for each market.

Governance becomes much harder once the organization is reviewing the system retrospectively while the business continues to depend on it.

The test for leadership

The conference poll gives leadership teams a direct test.

If a governance leader ordered a revenue-critical AI model to be suspended tomorrow, would that decision hold?

The answer reveals whether authority matches accountability. It also shows whether commercial leaders can override governance when performance is at risk.

Executive teams should know which AI systems are already business-critical, what evidence supported their approval, and who has the authority to intervene. Any uncertainty signals a weakness in the operating model.

Dr. Eva’s distinction between motion and progress matters here. Pilots and committees show activity. An organization proves its governance when leaders can enforce standards under commercial pressure.

She is direct about the leadership risk.

“The most dangerous executive in AI is the one who knows nothing but thinks he knows everything.”

A policy may create confidence, but the decisive test is whether its requirements still hold when revenue is at stake.