AI Agents Need Business Rules and Human Oversight to Deliver Real Value, Industry Leaders Warn

6 Sources

Share

As AI agent adoption accelerates toward 74% of companies by 2026, industry experts warn that deployment without proper business rules, human oversight and governance could lead to 40% of projects being canceled by 2027. Leaders from Get Covered, Meta and Alteryx share frameworks for managing AI agents like employees with clear accountability and verification mechanisms.

News article

AI Agent Deployment Requires More Than Technical Capability

As agentic AI moves from pilot projects to production systems, enterprises are discovering that technical sophistication alone doesn't guarantee success

1

. Deloitte's 2026 State of AI survey found that 23% of companies use agentic AI at least moderately today, with 74% expecting adoption within two years

1

. However, Gartner predicts more than 40% of agentic AI projects will be canceled by the end of 2027, signaling a critical gap between deployment ambition and execution capability

1

2

.

The challenge isn't whether AI agents can perform tasks, but whether organizations can manage them with the same rigor applied to human employees. Rick Folgmann, COO at Get Covered, operates 16 AI agents across more than 3 million rental units and emphasizes that "the technical questions turned out to be the easier ones"

1

. The harder question for each AI agent was determining who it reports to and what happens when it makes mistakes.

Business Rules Must Guide AI-Driven Processes

Successful AI agent deployment starts with tasks where business rules already exist in written form—statutes, contract clauses, insurance requirements, or compliance mandates

1

. At Get Covered, vendor certificate of insurance tracking became an ideal candidate because property teams spend roughly three hours per property every week chasing documents by email, and the work follows completely specified rules

1

.

McKinsey research shows that the small group of companies seeing significant bottom-line returns from AI adoption in enterprises—about 6% of respondents—were nearly 3x as likely to have fundamentally redesigned their workflows

1

. This redesign begins with documenting decision criteria explicitly. "If a new hire couldn't follow your description of a decision on the person's first day, an AI agent won't follow it either," Folgmann explains

1

.

At Alteryx, the VURA framework ensures AI workflows remain Visible, Understandable, Repeatable, and Auditable by design, setting standards that maintain consistency regardless of who interacts with the system next

5

. Without explicit business rules, AI agents make assumptions that become answers, potentially creating material misstatements in areas like revenue recognition or cost allocation

5

.

Human Oversight Determines Autonomy Levels

Autonomy isn't a fixed property of AI agents but should vary based on workflow risk and detection speed

1

. Get Covered configures each AI agent at different autonomy levels—a reminder about an expiring vendor certificate can send automatically, while responses to residents requesting emotional support animals never do

1

.

Leadership teams should evaluate every workflow an AI agent touches by asking: "If this action is wrong, how long until we find out, and what will it have cost by then?" Quick detection and low cost earn an AI agent more room to operate autonomously

1

.

Meta's approach to managing AI agents in infrastructure efficiency illustrates this balance. Tommy Tran, a software engineer at Meta, developed an agent platform that investigates performance problems and proposes fixes, but human engineers retain approval over changes to live systems

3

. The platform can reduce roughly 10 hours of manual investigation to about 30 minutes, but verification mechanisms ensure proposed changes receive proper review before implementation

3

.

Governance for AI Agents Requires Named Accountability

Every AI agent at Get Covered has a person ultimately responsible, the same way a team or budget does

1

. That person reviews error logs, decides when to widen or narrow autonomy, and answers for the agent's actions. "The model did it was never an acceptable answer" when explaining automated decisions to customers or regulators, Folgmann notes from his experience in financial services

1

.

Ownership also prevents AI agents launched independently by different teams from contradicting each other

1

. For solo businesses, where the US Census Bureau counted over 30.4 million nonemployer businesses generating nearly $1.8 trillion in combined revenue in 2023, the stakes are equally high despite smaller scale

4

.

Business Outcomes Matter More Than Agent Count

Across 200-plus workshops with enterprises in Europe and the Middle East, organizations getting real value from agentic AI aren't those with the longest agent inventory

2

. They identified business problems first and worked backwards, rather than starting from technology and searching for applications

2

.

The distinction between AI agents and agentic applications matters for measuring success. An AI agent helps with individual tasks, while an agentic application coordinates teams of specialized agents to deliver business outcomes

2

. "If you cannot name the KPI an AI agent is moving, you do not have a use case. You have a demo," industry experts warn

2

.

Value appears in measurable business outcomes: fewer payroll errors, faster shift scheduling, lower supplier sourcing costs, better win rates, or cash collected sooner

2

. For small businesses, KfW Research found sole traders carry the heaviest admin load of any business size, losing over 8% of working time to administrative processes—making workflow automation particularly valuable when properly implemented

4

.

Context and Institutional Knowledge Enable Effective AI Agents

Raw data doesn't explain the rules behind the numbers. AI agents need context about how different systems relate to one another to tackle open-ended assignments

5

. An AI agent calculating shipping costs might not factor in supplier agreement tariff exemptions or volume incentives tied to particular carriers without explicit context

5

.

Domain experts—accountants, analysts, and operations teams—hold the business rules and institutional knowledge that deterministic AI agents require

5

. Someone must map which systems communicate, audit legacy software and messaging apps, and capture the standards and exceptions that experienced employees apply instinctively

5

. This manual step has no shortcut, but it's essential for compliance and auditability.

Meta's approach combined finding software improvements and investigating performance problems within the same architecture by working with senior efficiency engineers to capture investigation methods

3

. These became reusable instructions, known as skills, that guide AI agents through particular problem types while keeping tools stable

3

.

Open standards like the Model Context Protocol, now stewarded by the Linux Foundation, give AI agents a common way to plug into business apps and data, reducing dependence on single-vendor integrations

4

. However, technical connectivity alone doesn't solve the governance challenge.

Risk Management Shapes Deployment Strategy

For businesses considering AI agent deployment, practical questions center on how much work can be delegated without requiring complete reconstruction and verification

3

. An AI assistant that drafts emails leaves text to review, but granting permission to send messages, update records, or change files introduces new complexity

3

.

Business email compromise, where scammers impersonate suppliers to redirect payments, cost US victims about $3 billion in 2025 according to the FBI

4

. Never allowing AI agents to change payment details or pay vendors without verification through separate channels represents essential risk management

4

.

Folgmann's advice for decision-makers heading into 2027 is to treat each AI agent like a new employee in a regulated role: write down its business rules, limit its scope, and assign it a manager who checks its work and discusses its performance

1

. Companies that establish this governance framework will be able to give AI agents more responsibility over time with evidence to justify it, while businesses that skip it will spend 2027 explaining why their software did something nobody approved

1

.

© 2026 TheOutpost.AI All rights reserved