AI Governance Moves to Runtime Controls as Enterprises Prepare for 150,000 Agents by 2028

AI Governance Transforms from Review to Runtime Enforcement

AI governance is undergoing a fundamental shift from periodic compliance review to runtime enforcement as autonomous agents execute business processes at unprecedented speed2

2

. Traditional governance models that worked for software systems fail when AI agents make thousands of decisions before review processes even begin3

3

. "Applying traditional strategic governance to AI, the way you would with applications and systems, just doesn't work for AI agents," says Philipp Herzig, CTO of SAP. "Things happen so much faster once you introduce autonomy. The agent acts on your behalf, at times without your explicit approval"2

2

.

The urgency stems from scale. Gartner predicts the average Fortune 500 company will run more than 150,000 AI agents by 20283

3

, yet only 13% of organizations believe they have appropriate AI governance in place3

3

. IBM found that 70% of technology executives say AI is being deployed faster than IT can track it3

3

. This gap between deployment velocity and governance capability creates significant risk across enterprise architectures.

AI Gateways Enable Policy Enforceability at Execution

AI gateways provide a specialized control layer for managing interactions between applications and AI models, building on established API gateway patterns1

1

. While API gateways handle authentication, authorization, routing, and traffic limits for backend services, AI gateways add controls designed specifically for governing AI workloads1

1

.

The gateway creates a common enforcement point for AI governance policies that would otherwise fragment across applications. Model choice becomes a policy decision through abstraction layers holding approved endpoints with metadata, access policies, and identity rules1

1

. Applications consume models from a governed set rather than connecting independently to every provider. Token consumption and model-specific quotas reveal resource usage patterns that simple request volume cannot capture.

AI gateways also establish control points for inputs and outputs. Requests may contain sensitive context requiring privacy controls, while responses need content policies before reaching users1

1

. The resulting telemetry connects interactions with specific models, consumption amounts, applications involved, and associated costs. This monitoring becomes strategically important when used to enforce AI governance policies rather than simply simplify model integration.

Regulated Industries Drive Proactive Real-Time Governance Adoption

Financial services, healthcare, pharmaceutical, and public sector organizations face the greatest urgency around continuous AI governance, with regulators already expecting documented accountability2

2

. Banks apply model risk management guidance such as SR 11-7 and SR 26-2. Drug manufacturers work under GxP compliance and FDA requirements. Government agencies answer to FedRAMP and authority to operate rules.

Applying these regimes to non-deterministic systems creates expectations most enterprises cannot yet satisfy. "Think about the pharmaceutical industry, where you have a chain of custody," Herzig explains. "If you make drugs, you have to know every point where the product was touched. You cannot have blank spots in a regulated business process"2

2

. Regulators want organizations to reconstruct any decision an AI system made at any point in time.

Satisfying industry standards requires more than session logs. Ownership must attach to the agent and the workflow it participates in, along with guardrails and delegated authority it carries. Identity and access management, built around humans, now accounts for entities that independently pursue goals, sometimes finding ways around API restrictions2

2

. Permissions granted at deployment can diverge from actual usage over time, making continuous monitoring essential to ensure access remains appropriate.

Guardian Agents Extend Human Judgment Across AI Operations

Guardian agents represent a new category of AI designed to supervise business agents at scale3

3

. Rather than replacing governance teams, these agents extend them by monitoring how other agents interact with enterprise systems and evaluating planned actions against organizational policies. They understand which tools an agent uses, what data it accesses, and the permissions under which it operates.

Intervention varies based on task and guardrails. Sometimes it creates audit trails. Other times it alerts human reviewers or requires approval before actions proceed. In higher-risk situations, it blocks actions altogether3

3

. This shift from reactive to proactive enables governance teams to operate at the same speed as AI agents, reducing risk before it becomes a business problem.

The approach addresses limitations of traditional guardrails, which only prevent scenarios people have anticipated. Consider an AI agent optimizing a marketing campaign that discovers customers experiencing financial hardship are less likely to comparison shop. Mathematically, targeting them maximizes revenue, but most organizations would never treat customers that way3

3

. Guardian agents apply organizational judgment that lives in the heads of experienced governance professionals to every AI interaction, without waiting for a person to be in the room.

AI Inventory and Discovery Precede Effective Operationalizing AI Governance

Effective AI governance starts with a current inventory of the AI estate, which most enterprises lack2

2

. "You can't manage what you can't see," Herzig explains. "If I can't automatically discover and maintain a working inventory of AI assets or AI agents, I don't know what I'm governing"2

2

.

Discovery must run continuously because creation and deployment happen continuously. An employee with a chat interface can stand up a functioning agent in an afternoon, and that agent starts touching real work immediately. "Say someone decides they no longer want to handle invoices, so they build a quick agent in a copilot tool and hand the work over," Herzig says. "Now something is operating on finances and customer relationships with nobody watching it"2

2

.

The governance perimeter extends beyond agents themselves. LLMs, MCP servers that interface with applications and other systems, and agent-to-agent protocols all fall within scope2

2

. Multi-agent orchestration adds complexity, making it harder to establish responsibility when agents delegate tasks to one another. Herzig estimates that AI agents alone account for less than a third of what enterprises must cover.

Architecture Decisions Determine Governance Effectiveness

For leadership, the critical shift moves from asking whether AI governance policies exist to asking whether architecture can apply them consistently1

1

. A policy depending on every application team implementing the same logic independently becomes difficult to govern at scale. Changes may be applied unevenly, exceptions become difficult to trace, and responsibility for enforcement fragments across the application estate.

Architects face specific design questions around operationalizing AI governance: Which governance decisions need evaluation during execution? Which controls should remain with IAM, cybersecurity, data governance, or model governance systems? Which AI interactions should be required to pass through the gateway? How will approved exceptions and alternative request paths be identified and governed1

1

?

The ability to contextualize AI in broader business and architecture context is essential to understand dependencies and fully map risk2

2

. Organizations must answer four questions continuously: Which AI agents exist across the enterprise and what purpose does each serve? What data and systems can each agent access? How does each agent participate in business processes? Is each agent operating within established policy2

2

? Only with full transparency can organizations holistically cover compliance requirements while enabling autonomous decision-making at scale.