Why enterprises may soon pull the plug on autonomous AI agents

Enterprises have spent the past year racing to deploy AI agents across customer service, software development, operations and internal workflows, hoping autonomous systems will unlock the next productivity leap. But many companies are now discovering that giving AI agents access to enterprise systems without proper governance guardrails may create more risk than value.

Research firm Gartner predicts that by 2027, nearly 40% of enterprises will either demote or decommission autonomous AI agents after governance failures surface in production environments.

The warning comes as enterprises shift from AI copilots that assist employees to “agentic AI” systems capable of independently taking actions, interacting with enterprise applications and executing workflows.
“Enterprises are treating AI agent governance as binary — either locked down or fully trusted — and that is the root cause of failure,” said Shiva Varma, senior director analyst at Gartner.

According to Gartner, organisations are making a fundamental mistake by applying the same governance framework to every AI agent regardless of its autonomy level or system access. This often creates two extremes — simple AI tools become overregulated and unusable, while highly autonomous agents receive insufficient oversight despite being able to execute sensitive actions.
That shift is raising the stakes for CIOs, CISOs and governance teams.

Unlike traditional software, autonomous AI agents can behave unpredictably, adapt dynamically and make decisions at machine speed. A poorly governed agent connected to financial systems, customer databases, or cloud infrastructure could trigger operational disruptions, compliance breaches or security incidents before human teams intervene.

Industry executives say enterprises are underestimating how quickly these risks escalate once AI systems move beyond read-only assistance into action-oriented autonomy. In aviation, for instance, Air India has deployed its AI agent “Maharaja” to manage multilingual customer interactions as part of its digital transformation efforts. While such deployments improve efficiency, they also increase the need for tighter oversight around data access, escalation workflows and decision accuracy.

Similarly, Indian startup M37Labs recently introduced an enterprise-grade “agentic AI” platform focused on governance and data sovereignty, highlighting how governance itself is becoming a competitive differentiator in enterprise AI.

The concern is that enterprises are scaling AI agents faster than their governance models can evolve. In a separate interview, Prakash Thekkatte is Senior Vice President - Software Engineering, India at Salesforce, argued that organisations increasingly require orchestration layers to manage AI workflows, approvals and policy enforcement consistently across departments. As enterprises deploy multiple AI agents simultaneously, governance becomes essential to prevent policy drift, workflow conflicts and operational failures.

Dhundia, principal at management consulting and technology firm ZS, further warned that responsible AI frameworks can no longer be “added later” once systems become autonomous and workflow-driven.
Gartner argues that enterprises should adopt layered governance tied to autonomy levels rather than a single universal policy. At the lowest level are “observe” agents that summarise or retrieve information with read-only access and therefore require lightweight controls focused on authentication and monitoring.

The risk profile changes with “advise” agents that generate recommendations for humans to approve. Here, Gartner warns of “automation bias” — the tendency of employees to overtrust AI-generated outputs even when they may be inaccurate or hallucinated.

The highest concern lies with fully autonomous agents capable of independently executing actions within enterprise systems.
“At this level, human review is effective only if it remains a meaningful control,” said Varma. “Without strong security testing, clear approval workflows with audit trails, and agent-specific incident response procedures, approvals can degrade under time pressure or approval fatigue.”

The governance challenge is also becoming a cybersecurity issue. Security researchers warn that autonomous AI agents can bypass traditional identity and access frameworks because they interact dynamically across systems, APIs and enterprise applications.
As enterprises expand AI access across ERP systems, cloud infrastructure and developer environments, the challenge is no longer simply deploying AI agents quickly, but controlling how far they are allowed to act.

For CIOs and boards, the next phase of enterprise AI may depend less on model sophistication and more on governance maturity. The companies that succeed with agentic AI could ultimately be the ones that treat governance not as a compliance afterthought, but as foundational infrastructure for the autonomous enterprise era.