AI Agent Security: Traditional Controls Fall Short as Autonomous Systems Expose New Risks

Box's approach to AI agent security | VentureBeat

Identity and permissions are no longer enough to secure enterprise AI agents. They govern what an agent can reach, not how it behaves once it starts working on its own, and an autonomous agent can turn legitimate access of enterprise data into unintended action in seconds. That gap is pushing enterprise AI security from just governing access toward a layered approach that includes governing execution, says Heather Ceylan, chief information security officer at Box. "Access controls and permissions are the foundation, but the challenge is they were designed for humans," Ceylan says. "Permissions are still the foundation, but you have to think about how the agents get their permissions scoped as well." Access controls were built for a slower, more forgiving world, where a human employee with lingering permissions to a decade-old folder will rarely go looking for any data inside it, or even remember they have access at all. But an agent will explore all of its permissions, operating at a scale no person can match and surfacing forgotten misconfigurations and stale permissions far faster than a human ever could. Identity and scoped permissions remain the first layer of any credible defense, however, because agents raise the stakes, clean access hygiene is more valuable than ever. A steady drip of incidents in recent months underlines the potential for major damage, where models have slipped the sandboxes they were supposed to stay contained within, reached systems they weren't scoped to touch, or read content they were not permitted to access. While the specifics vary, the overarching threat remains the same: an agent found a path through the data that it could reach, and took it. Access controls alone can't secure autonomous AI agents Complications crop up when agents are handed broad standing permissions to match broad workflows. An agent might legitimately need to call fifty tools across twenty different actions over the life of a task, reading and writing folders that span every department in the company. But granting all of that access at once inflates the blast radius of possible damage to data and content if a single step goes wrong. A stronger permissions model provides access only in the moments that it's needed, Ceylan says. "You need permissions that change based on what the agent has been asked to do, when it needs to take that action," she explains. "If it's taking one step and only needs two tools, it should be scoped to only those two. When you narrow permissions to the task in front of the agent, the number of ways any given step can misfire shrinks with it." From governing access to governing execution Tight security is also no longer tied to whether an agent has access to specific data, but whether it should take a specific action on a specific step, which is the difference between a standing access grant and a bounded one. In other words, an agent may hold legitimate rights to a finance folder but still have no business writing four thousand of those files to a new location, even if it's told to do so. "An employee with access to payroll data they were never meant to keep could be instructed to pull the payroll records and write them to a public shared folder, publishing the entire company's compensation in a single move," she says. "Every access check passed, but the behavior still has catastrophic consequences." Prompts alone can't reliably govern agent behavior because instructions can change, agents can absorb injected instructions, or they can be steered by files they read along the way. Durable controls need to live one level down, with the tool calls themselves, as well as the content those calls act on. Deciding in advance what an agent is permitted to execute, regardless of how its prompt is manipulated, keeps behavior inside fixed bounds that no clever input can talk it out of. Legacy content platforms weren't built for AI agents Most of what an agent touches inside a company is unstructured content, including contract, policies, customer records -- the accumulated files that make up the overwhelming majority of corporate data. Much of it still sits in systems that were designed for human filing habits: network drives, aging ECM platforms, and many SaaS tools. Those systems were built to ask the question, does this person have permission? and they answer it with folder-level access that hasn't been audited for years. The legacy platforms don't have metadata for an agent to reason over or classification for an enforcement layer to act on, and their logs aren't detailed enough to show what an agent read. Bolting an AI connector onto that stack doesn't fix any of it, it just hands agents the same blind spots, only at machine speeds, Ceylan says. "Every agent action eventually resolves to content," Ceylan says. "If the content layer can't tell you what it's holding, who it belongs to, and what should never leave it, there's nothing underneath your controls." How to decide which AI actions need human approval Two years ago, everyone assumed security would always mean including humans in the loop. Building and living with agents has changed that assumption, Ceylan says, and Box now sorts actions into three tiers: fully autonomous actions, monitored actions and high-risk actions that require human approval. * Fully autonomous, reserved for actions that are reversible, bounded, logged, and free of untrusted input, and where a mistake doesn't cost too much. * Monitored actions, once a team has built enough confidence in an agent and paired it with alerting and rollback that can catch and undo a problem in flight. * Irreversible, high-risk actions that are always routed through a person -- for instance, when an agent that wants to delete a large number of files or wipe the primary folder in a structure. Each team must calibrate the lines between these layers to match its own risk tolerance. Box's approach is to put controls in the platform rather than the workflow, enforcing protections such as data classification, labeling and expiration without requiring a human checkpoint for every action. "The right configuration should be enforced at the outset, instead of blocking an action at the end," Ceylan says. Building trust in AI agents through behavior over time An organization can make an access decision once, but trusting an agent's behavior should only happen over time, observing how the agent runs, collaborates, and builds on the outputs of other agents. Published frameworks tend to lag the technology, or are too abstract to implement; instead Box works from a smaller set of principles: tightly scoped agent identities and actions, clear expectations about what can be rolled back, three approval tiers, and a way for teams to test and iterate quickly. "The sanctioned path has to be the fast path, because when teams aren't given a safe way to experiment, they tend to route around the controls entirely," Ceylan says. "A security leader's job is to offer a way to move quickly without stepping outside the guardrails." Sustaining trust in an agent depends on being able to see how it actually behaves. But traditional monitoring tools weren't built for the job; the behavioral baselines that underpin user and entity behavior analytics were calibrated to human activity, and suspicious agent behavior doesn't necessarily look like human activity. And while establishing a useful agent baseline begins with logging, many agents start out as experiments, so their actions never make it into logging infrastructure. Plus, important signals are rarely individual easily flagged actions, but chains of activities across systems, as one agent's output becomes another's input, while the detections for that pattern are still being designed. Governing agents, then, requires visibility not just into what they can access, but what they actually do, and that visibility has to live where the content does. Sponsored articles are content produced by a company that is either paying for the post or has a business relationship with VentureBeat, and they're always clearly marked. For more information, contact [email protected].

AI agents need their own identity before they need a gateway

Enterprise AI has entered a new era. Organizations are rapidly moving beyond assistants that answer questions to autonomous agents capable of reasoning, invoking tools, accessing enterprise applications, coordinating with other agents, and completing multi-step business workflows with minimal human intervention. This shift represents a fundamental change in how software operates. Traditional applications execute predefined logic written by developers. AI agents, however, dynamically determine how to achieve an objective. They decide which tools to use, which APIs to call, what information to retrieve, and how to sequence actions based on context. That flexibility unlocks enormous business value, but it also introduces a new class of security risks. Much of today's AI security discussion focuses on prompt injection, model vulnerabilities, and data leakage. These are important concerns, but they represent only part of the challenge. Once an AI agent has successfully authenticated and begins acting autonomously, traditional security controls provide very little visibility into whether it continues to operate safely. This is where enterprises need to adopt a new security mindset: runtime trust. Authentication establishes identity, not trust Enterprise security has traditionally relied on three foundational questions: Who are you, what can you access, and what actions are you authorized to perform. Identity providers, multi-factor authentication (MFA), role-based access control, and zero trust architectures answer these questions effectively for human users and conventional applications, and NIST's zero trust guidance remains a solid reference point for how those principles are meant to work (NIST SP 800-207). AI agents introduce a different problem. An AI agent may legitimately authenticate using an enterprise identity, receive valid API credentials, and be granted access to systems like Microsoft 365, ServiceNow, Salesforce, or GitHub. From an identity perspective, everything appears correct. The real challenge begins after authentication: During execution, the agent continuously reasons, interprets objectives, invokes tools, retrieves information, and adapts its behavior based on new context, and security teams must determine whether those actions remain aligned with the user's intent and organizational policy. Authentication verifies who an AI agent is. Runtime trust continuously verifies what it is doing. Enterprise AI is becoming an autonomous workforce Modern AI agents increasingly interact with large language models (LLMs), Model Context Protocol (MCP) servers, retrieval-augmented generation (RAG) systems, vector databases, enterprise APIs, SaaS platforms, and internal knowledge repositories, as well as other AI agents. This interconnected ecosystem enables sophisticated automation but dramatically expands the attack surface: A single compromised tool, poisoned knowledge source, overly permissive API, or manipulated prompt can influence downstream decisions across an entire workflow, and unlike traditional software, these risks evolve during execution rather than being fixed at deployment. That expanding surface is exactly what a handful of runtime threats exploit. Goal drift happens when an agent begins with a legitimate objective but gradually deviates from the user's original intent while attempting to optimize outcomes. An agent tasked with preparing a customer report, for instance, might autonomously retrieve unrelated confidential information because it incorrectly determines that additional context would improve the response. Excessive tool invocation is what happens when autonomous agents with access to numerous enterprise tools call unnecessary APIs, modify configurations, access sensitive repositories, or perform administrative actions simply because the model believes those actions are useful, absent any runtime controls to stop it. Memory poisoning exploits the persistent memory that improves personalization: Attackers can intentionally insert misleading instructions into long-term memory or retrieval systems, causing future decisions to be influenced by malicious or outdated information. Context manipulation takes advantage of how heavily LLMs depend on context: If attackers influence retrieved documents, system prompts, conversation history, or external data sources, they can indirectly steer autonomous behavior without ever compromising the underlying model. MITRE's ATLAS framework catalogs this kind of adversarial behavior against AI systems in useful detail. Multi-agent amplification emerges as organizations deploy specialized AI agents that collaborate: If one agent behaves incorrectly, downstream agents may trust and amplify those actions, creating cascading failures across enterprise workflows. Introducing runtime trust Runtime trust extends security beyond authentication by continuously validating AI behavior throughout execution. Rather than assuming authenticated agents remain trustworthy indefinitely, it continuously evaluates whether autonomous decisions remain aligned with organizational policy. A runtime trust architecture rests on several complementary capabilities. Intent validation evaluates, before executing sensitive actions, whether proposed behavior still matches the user's original objective: Is this action necessary? Is it expected? Does it exceed the requested scope? Would a reasonable human perform the same action? Behavioral monitoring observes tool usage, API activity, reasoning patterns, execution frequency, delegated actions, and abnormal workflows, so unexpected behavior becomes immediately visible rather than remaining hidden inside model reasoning. Policy enforcement means enterprise policies govern what AI agents can do, not merely what they can access -- blocking financial transactions above approval thresholds, preventing privilege modifications, restricting administrative operations, limiting sensitive data retrieval, and requiring approval for high-risk actions. These controls function much like application firewalls for autonomous decision-making. Least-privilege execution means AI agents receive only the capabilities necessary for the current task. Instead of granting permanent access to dozens of enterprise tools, organizations should dynamically issue short-lived permissions based on runtime context, an approach that OWASP's guidance for agentic applications increasingly emphasizes (OWASP GenAI Security Project). Human oversight recognizes that not every decision should be autonomous -- high-impact operations, including financial approvals, identity changes, regulatory actions, or customer-impacting decisions, should require explicit human confirmation before execution. Protecting the enterprise AI ecosystem Runtime trust also extends beyond individual agents. As MCP adoption accelerates, enterprises should verify trusted servers, authenticated tools, approved capabilities, monitored interactions, and policy enforcement. RAG knowledge repositories require document integrity, source validation, access control, retrieval auditing, and poisoning detection. Persistent AI memory should implement lifecycle management, expiration policies, integrity verification, access logging, and sensitive data protection. Building operational visibility One of the biggest challenges in enterprise AI is observability. Security teams need visibility into why an agent selected particular tools, which data influenced its decisions, how it reached its conclusions, what actions it executed, whether policies were triggered, and which safeguards prevented unsafe behavior. Runtime logging, audit trails, and behavioral analytics are becoming essential components of enterprise AI operations, not optional add-ons. A practical roadmap Organizations do not need to rebuild existing security programs. Instead, they should extend them by incorporating runtime trust into existing governance processes. Practical first steps include inventorying AI agents and their capabilities, applying least-privilege access to tools and APIs, classifying high-risk autonomous actions, implementing runtime policy enforcement, monitoring behavioral anomalies continuously, protecting memory and RAG data sources, requiring human approval for critical operations, and integrating AI runtime telemetry into existing SOC workflows. Looking ahead Enterprise AI will continue evolving toward increasingly autonomous systems capable of collaborating, planning, and executing complex business processes. Security strategies must evolve alongside them. The question is no longer whether an AI agent successfully authenticated. The more important question is whether it continues to behave safely throughout its entire lifecycle. Organizations that adopt continuous runtime governance today will be significantly better positioned to deploy autonomous AI responsibly, reduce operational risk, and build the confidence necessary for large-scale enterprise AI adoption. The future of AI security will not be defined solely by stronger models or better authentication. It will be defined by our ability to establish, measure, and continuously verify trust while intelligent systems are making decisions in real time. Ravindra Annam is a cyber security architect. Welcome to the VentureBeat community! Our guest posting program is where technical experts share insights and provide neutral, non-vested deep dives on AI, data infrastructure, cybersecurity and other cutting-edge technologies shaping the future of enterprise. Read more from our guest post program -- and check out our guidelines if you're interested in contributing an article of your own!

Four safeguards to stop your AI agents from going rogue

Artificial intelligence agents are moving from experimentation to production, and with this shift, the stakes are rising. A coding agent at PocketOS recently deleted an entire production database. An agent at Meta exposed sensitive user data for two hours. An Instagram support chatbot allowed hackers to hijack thousands of accounts. And last month, researchers tricked a GitHub agent into leaking private repository data. In each case, the agent did what it was built to do; what failed was everything around it. Intelligence is advancing faster than organizations can safely deploy it. Providing enough context and controls to ensure an agent produces accurate results and doesn't overstep its bounds is the core quandary. This difficulty helps to explain why model vendors have been investing in partnerships to help their customers make AI work, and why "forward-deployed engineer" has become the hottest job in tech. Models excel at pattern recognition, but safe deployment demands understanding thousands of disconnected systems, data structures, and compliance rules that power businesses' operations. Closing this gap doesn't require an army of consultants. It requires an architectural framework with four characteristics, each one helping to convert probabilistic outputs from an AI system into verifiable decisions your business can trust. Sense An agent is only as reliable as the information it works with. If a policy changes, a system goes down, or a customer's status shifts, and that update doesn't reach the AI in time, every decision downstream is going to be made against facts that are out of date. Sense is the architectural layer that keeps that picture current. Building this means connecting to data wherever it lives, across departments, systems, and clouds, rather than requiring everything to be centralized, which is why enterprise AI projects often stall before they start. It also means treating this data as a live signal, not a snapshot. The system needs to notice material changes and pass that forward in real time, not on the next scheduled sync. Decide Data shows what's happening now. It doesn't show what happened the last time someone made a similar decision, or why that outcome mattered. This difference separates an agent that can "see" your business from one that understands how it runs. Building this judgment into AI means grounding it in an organization's own decision history and policies, not just its live data. An agent handling a routine request should have access to how the last 20 similar requests were resolved, and what happened as a result. In the Meta incident, an internal agent gave an engineer flawed technical guidance that led to data being exposed. With better context, drawn from how similar historical changes played out, this outcome might have been avoided. Act An agent that makes recommendations still leaves the work to people. Maximizing the return on AI investments requires agents that can execute, not just advise. Building one capable agent is manageable. Coordinating several into a workflow takes many layers. The system needs to pass context between steps, apply consistent policy at each one, and stay in sync as underlying systems change, all without losing the governance that keeps an agent from acting outside its lane. This orchestration layer enables agents to work in concert and reliably complete end-to-end tasks. Secure Autonomous tools need the same access discipline a company already applies to its employees: scoped identity, permissions limited to the specific task, and a clear audit trail. In the Instagram incident, tighter scoping and a real-time check on what the bot was authorized to do might have caught the error. Every agent needs a permission set tied to its specific role and logging of every action it takes. They also require a kill switch -- a way to cut off an agent's access the moment something looks wrong. All the identity and access management best practices your company already knows, now need to be extended to non-human actors. The upside is still real None of those prerequisites detracts from how far models have come and what they can achieve. Models are advancing rapidly, and their capabilities are real. But intelligence was never the hard part. Connecting it safely to a business -- its systems, its rules, its history of what's worked and what hasn't -- is where a lot of AI investments lag. Agentic AI has real upside for the businesses that get there. Sense, decide, act, and secure are how you get there safely. Amit Zavery is president, chief product officer and chief operating officer of ServiceNow Inc. He wrote this article for SiliconANGLE.

AI agents that pass authentication can still drift, expose data, or get memory-poisoned

There is a clear repeating trend in agent deployments: The gateway is the first control teams reach for, but it is the one they are least ready to run. This is because gateways sit on top of identity and attribution layers that are mostly not there. The first layer of risk is not hypothetical. In June, CISA added a LiteLLM flaw to its Known Exploited Vulnerabilities catalog after attackers were caught abusing it in the wild. The bug ran commands on the host through the gateway itself, and chained with a second flaw it required no credentials. It was one of seven common vulnerabilities and exposures (CVEs) disclosed in that single AI gateway in a month. This is the layer many enterprises reach for first to secure their AI agents. When considering secure agent architecture, gateway controls should not be the first control. They should be the fifth. Most models on the maturity of agent security describe the controls a company will need in the future. They tend to miss, from my experience, the more difficult problem of describing the brownfield scenario: In what order should these controls be layered in conjunction with an identity and access management system that is already in place? If the control plane is unaware of which agent is acting, who delegated the work, what task the agent is to perform, and what credentials are being used, then the context is incomplete. A gateway may block clear policy violations, but will struggle to distinguish a justified action from one that is technically permissible but operationally inappropriate. The pattern of failure is clear when sequencing these controls for agent production deployments: Enforcement is taken early, while the identity and attribution context it depends on has yet to be developed. Agent security functions as a dependency chain, with each control depending on context generated upstream. The wrong starting point Think about routing agent traffic via a new runtime gateway. A finance-reconciliation agent tries to alter a record in production. The gateway authenticates the user token and checks the API call. What it can't observe is that the request is agent-initiated, that the agent is executing a more limited function, or that the request is part of a tool chain invoked by an untrusted artifact. The credential is valid. The API call is permissible. The action contradicts the purpose of the delegation. The gateway is there, but its set of supports seems absent, so a costly control is applied to a very small part of the whole picture. Limiting an agent's privileges to those of the human principal is useful so the agent does not exceed the person it serves. However, having a privilege ceiling does not create separate attribution. Twenty agents might operate under a single person's permissions and still need unique identities, audit logs, behavior profiles, and revocation paths. Dependency-gated deployment I call this process dependency-gated deployment. Upstream exit tests must be satisfied before any downstream control is considered operationally complete. Concurrent development of downstream controls is permissible. Here are the six gates, and the proof that they work: The six dependency gates for the agent security controls. Each control is contextualized by the gates above it. From the author's analysis of production agent deployments. Start with the agents you can actually name To begin, recognize the production agents in open-source frameworks, cloud offerings, SaaS services, and developer tools. For each, record the owner, responsibility, lifecycle stage, allowed tools, data domains, and sources of credentials. Bypass this step, and the organization will lose the first hour of incident response while they figure out what should have been obvious. The inventory identifies the asset that every control thereafter governs. An agent needs its own identity, but it cannot lose the human behind it An agent should not be buried in a developer token, a shared service account, or a human session. Simply knowing the caller is an agent is not sufficient. The control plane requires additional delegation context: Who delegated the work, what specific task the agent was instructed to execute, and which resources the agent needs the authority to access. Identity specifies which actor placed the call. Delegation is the answer to whose authority it acts, and for what reason. Once that connection is cut off, the downstream logs attribute the reconciliation agent to the employee whose token it borrowed, and every action it takes is attributed to someone who did not start it. Shrink authority before you inspect behavior Once an agent can be identified, capabilities should be limited. Access restrictions should be time-bound to the task and limited to the tools and resources required to perform the task. This can be implemented using identity access management (IAM) features such as workload identity, token exchange, conditional access, and time-bound entitlements which the organization already possesses. With regard to the 2026 Teleport study involving 205 security leaders, the access scope surpasses the predictive capacity of industry, maturity, or self-assurance concerning predicting AI-related incidents. For example, organizations with over-privileged AI reported a 76% incident rate, whereas AI incidents occurred in 17% of organizations under the least privilege. This indicates that access scope in the dependency chain is more important than context-aware runtime enforcement. The primary principle is monotonic delegation. Every transfer of responsibility must preserve or diminish authority; under no circumstances should it increase authority. For the reconciliation agent, this means an agent who can view one ledger as opposed to one who inherits the employee's access to all systems the employee can access. Fix attribution before automating enforcement Most audit stacks can capture what resource was accessed and which credential allowed access. In the agent deployments I have reviewed, this is the most commonly missed gate. Prior to utilizing an adaptive runtime policy, link any relevant tool invocation to the agent identity, initiating principal, task id, parent action, and outcome. After doing so, examine the telemetry: For one completed task, see if you can track down the initiator, the agent who executed it, the authority under which the action was taken, the tools utilized, and the outcome. In regulated environments, oversight that is not attributed cannot be justified. Now the gateway earns its keep The gateway can use registered identities, explicit delegation, scoped credentials, and attributable telemetry to question if this agent is authorized to perform this action, for this principal, within this task, involving this resource. Although the user's credentials may provision write access to the finance-reconciliation agent, the gateway has situational context and so determines that it is out of scope. This is control's point of greatest value. The most stringent controls should be applied at irreversible boundaries -- payments, access policy changes, deletions, modifications of the production environment, and data exports. Detection and the kill path come last Behavioral baselines are developed last because distinguishable and attributable agent activity must be established to set a standard. Then, security teams are able to identify anomalous patterns of tool usage, unexpected cross-domain access, and deviations from their assigned tasks. Containment is more than just the disabling of a single directory object: A proper kill path entails disabling the agent's identity, invalidation of active and derived credentials, blockage of tool activation, termination of active tasks, and isolation of the workload that contains the agent. Start without replacing your IAM Designing a whole new identity program is unnecessary. If the existing identity provider doesn't treat agents as native object types, begin with an authoritative registry linked to the existing workload identities. Following this, extend agent and task identifiers as trusted execution contexts, implement short-lived credentials to mitigate inherited privileges, and include those identifiers in tool-call logs for subsequent gateway ingestion. The dependency model remains unchanged as vendor support matures. Control gaps are measurable. In Okta's 2026 survey, only 34% of executives said their organization always applies the same level of security rigor to its agentic workforce as to its human workforce. The last control from the chain cannot be applied first to close that gap. What to do in the next 30 days Begin with 10 production agents. For each one, identify the owner, purpose, approved tools, and credentials. By now, you should have the beginnings of an agent registry and perhaps your first insights on governance. Test attribution. Find out if IAM and logging can tell each agent apart from the human or service that delegated the task. If this kind of differentiation is not possible, a gateway would be operating without any visibility. Reconstruct one completed agent task within an action chain, from start to finish, including downstream effects. Wherever the chain breaks is where your deployment falls short. Adding downstream enforcement ahead of required context breaks agent security. Maturity models describe the destination. A build order gets you there without breaking production along the way. Nik Kale is a principal engineer specializing in enterprise AI platforms and security. Welcome to the VentureBeat community! Our guest posting program is where technical experts share insights and provide neutral, non-vested deep dives on AI, data infrastructure, cybersecurity and other cutting-edge technologies shaping the future of enterprise. Read more from our guest post program -- and check out our guidelines if you're interested in contributing an article of your own!

Stopping the AI Agent Actions No Rule Could See Coming

Check Point introduces a new class of contextual AI protection that understands an agent's full context, intent and behavior across multiple steps, and prevents harmful actions before they execute. AI agents are already operating inside the enterprise. Coding agents write and execute code, interact with repositories, access cloud infrastructure and invoke tools. Workforce agents process documents and messages, retrieve enterprise data and complete business workflows. As these systems act with greater autonomy, security has to look beyond malicious prompts and individual payloads to the outcomes an agent's actions can create. Harmful behavior does not always begin with something recognizably malicious. It can emerge naturally from a sequence of otherwise legitimate actions, where the risk becomes visible only in context. Check Point's approach brings the full context of the agent's activity into the security decision. The protection continuously evaluates the full course of an agent's activity, including the original user intent, the information the agent has encountered, the actions it has already taken, the policies that apply and the action it is about to execute. It connects those signals in real time and prevents actions that are harmful, unauthorized or inconsistent with the intended task. This matters because agentic risk often does not appear in a single prompt, response or tool call. It emerges from context. Seeing what isolated guardrails cannot Most AI security controls are designed to recognize a specific type of risk. One may detect prompt injection. Another may identify sensitive data. Another may flag off-task behavior. These controls remain important, but they depend on knowing what to look for. Check Point's contextual protection adds a fundamentally different capability: it can recognize harmful agent behavior without requiring a predefined rule or signature for that specific behavior. By understanding the task and the agent's activity as a whole, it can identify when an otherwise legitimate action creates a harmful outcome in context. Consider a coding agent working with production systems. It accesses production logs as part of a legitimate task. Later, it stores information from those logs in a working variable. Then it attempts to use that variable while creating a ticket in a public-facing system. Each step can appear reasonable in isolation. Together, they create a data leak. Check Point's contextual protection understands the relationship between those actions and blocks the final step before the sensitive information leaves its intended environment. The risk was not hidden inside one action. It was spread across the sequence. Understanding intent, not just instructions The same contextual understanding allows Check Point to identify when an agent's behavior begins to diverge from what the user actually asked it to do. In one real-world case, an autonomous coding agent operating in auto mode attempted to upload files directly to an S3 bucket. The user had not requested the upload, and the expected development process required the files to move through GitLab. Uploading a file to S3 is not inherently dangerous. In another workflow, it could be exactly the right action. Here, it was not. Without relying on a predefined rule or signature for this specific behavior, the protection understood the broader context of the task, recognized that the action did not fit the user's intent, and prevented the upload before it occurred. This was not a scripted attack scenario. It was genuine unexpected behavior from an autonomous coding agent during real use. That distinction captures one of the most important challenges in agent security. Agents do not simply execute predefined commands. They reason about how to complete objectives, choose tools and determine intermediate steps. That flexibility is what makes them powerful, particularly in development environments, but it also creates room for actions that were never explicitly requested and may fall outside the expected process. A technically valid action can still be the wrong action. "Agents are useful because they find approaches we did not anticipate. With a ruleset, protection is limited to what we anticipated when we wrote the rules. A security model that evaluates the full task checks each action against what the user asked for and prevents the ones that do not fit. That is what customers are asking us for, and why this capability matters now." Ofir Israel, VP AI Security R&D, Check Point Preventing harmful agent actions Contextual understanding also adds another layer of defense against indirect prompt injection. A user may ask an agent to summarize an email, document, webpage or message. Hidden inside that content is an instruction planted by an attacker telling the agent to retrieve credentials, export information or perform some other unauthorized action. The agent sees both the legitimate content and the attacker's instructions. Check Point sees the larger context. The user asked for a summary. The resulting attempt to retrieve credentials or send information elsewhere does not belong to that task. The protection recognizes the mismatch and prevents the action. Rather than relying only on detecting the malicious instruction itself, the protection can also recognize the harmful action it attempts to trigger. The same approach applies when the source of the risk is not malicious. An agent may attempt a destructive command that has no reasonable connection to its task. It may try to grant itself broader permissions without the required approval. It may attempt to send confidential information to a destination that conflicts with enterprise policy, even when the original instruction came from a legitimate user. These situations are different in cause, but similar in the way they appear at the point of action. The individual command may be valid. The tool may be approved. The user may be legitimate. The context reveals whether the action belongs. From individual signals to the full agent story The innovation lies in bringing those signals together continuously. Check Point evaluates the agent's activity across the flow of work and maintains the context needed to understand how the current action relates to what came before it. That includes user intent, the information the agent has seen, prior tool activity, the state of the task and applicable policy. The result is a much richer security decision at the moment the agent is about to act. Prevention without slowing the agent down For autonomous systems, the timing of the decision matters as much as the quality of the decision. Check Point evaluates the intended action before execution and prevents harmful activity while the outcome can still be avoided. With decisions made in approximately 50 milliseconds, the protection enables real-time enforcement while preserving the speed and productivity users expect from autonomous agents. The practical result is straightforward. Sensitive information remains inside the environment. Unauthorized uploads never occur. Destructive operations do not execute. Unexpected permission changes are stopped before they take effect. The security decision happens at the last responsible moment, with the full context needed to make it correctly. Built for autonomous AI Coding agents provide some of the clearest examples of why this capability matters today. They routinely operate across code, repositories, terminals, cloud platforms and production environments, often with enough autonomy to make meaningful decisions about how tasks are completed. But the same security challenge extends across the broader agent ecosystem. Enterprise agents are reading email and documents, accessing sensitive business data and interacting with SaaS applications. Multi-agent systems are beginning to pass information and instructions between agents, creating chains of activity where the context of one system can directly influence the actions of another. Across these environments, the security requirement is consistent: understand what the agent is doing in the context of what it was supposed to do, and prevent the action when those two no longer align.

Defending Enterprise Applications Against Agent-Era Threats

Join the DZone community and get the full member experience. Join For Free The rise of autonomous AI agents within business software demands a fresh approach to security. Unlike earlier chatbot tools, modern agents act with real privileges, such as updating databases, calling microservices, composing and even executing code, or triggering workflows on their own. This shift expands the blast radius of any flaw or compromise. As one Microsoft analysis observes, today's AI agents "can update database records, trigger enterprise workflows, access sensitive data, and interact with production systems all autonomously." In practice, that means a mistake or exploit can have immediate operational impact instead of just a reputational cost. With agents in the loop, input manipulation becomes especially dangerous. Prompt-injection attacks let adversaries commandeer an AI by feeding it malicious instructions in user inputs or hidden in external data. A carefully crafted prompt or document can cause an agent to reveal secrets or perform harmful actions. These manipulations can be direct (an attacker's text overriding the agent's instructions) or indirect (for example, hidden commands embedded in HTML or metadata that the agent ingests). By definition, even inputs imperceptible to humans can subvert the model, forcing it to break safety rules. In effect, prompt injection can trick an AI into disclosing internal prompts, executing arbitrary commands, or making unauthorized changes. Protect AI Agents With Layered Security Controls Defending against prompt injection requires layered controls. It is not enough to trust the LLM's built-in safeguards. The application must sanitize and constrain every input. For example, the OWASP GenAI guidelines recommend semantic filtering of user inputs and strict output validation. In practice, this often means cleaning or escaping suspicious tokens in the prompt, enforcing clear response schemas, and even tagging or quarantining untrusted data before it reaches the model. Developers should also build resilience into AI calls, for example by wrapping each agent invocation in a circuit-breaker or retry mechanism so that anomalous behavior triggers a safe fallback rather than a cascade of errors. In this example, strips any suspicious content from the prompt, and the circuit-breaker ensures repeated failures lead to a controlled fallback. The method logs the failure and returns a safe default response, preventing a hijacked prompt from causing uncontrolled retries or side effects. Embedding such patterns helps contain injected instructions and makes anomalies visible for audit. Agents also expand supply-chain and data-poisoning attack surfaces. AI applications often depend on third-party models, libraries, or datasets, each of which could harbor backdoors. In a real incident, attackers compromised an open-source Python package used in a model's pipeline, effectively inserting malicious logic into every system that imported it. To guard against this, organizations must treat AI dependencies as critically as any library or service. Models and data should come from verifiable, signed sources, and teams should maintain an AI-focused Software Bill of Materials (SBOM) tracking each model and dataset. Regular scans of model files and packages (for example, by verifying cryptographic hashes or digital signatures) can detect tampering before models reach production. Another insidious vector is agent memory poisoning. Unlike stateless microservices, AI agents may accumulate knowledge across sessions or tasks. If an adversary can insert malicious "memories" or biased information into that knowledge base, the agent may repeat or amplify harmful logic over time. Researchers have shown that injecting only a few hundred carefully crafted documents into a training or retrieval database can reliably hijack a model's outputs in specific domains. In an enterprise, this might translate to a support chatbot that starts rejecting valid requests or approving fraudulent transactions because its knowledge was skewed. Mitigations include thoroughly vetting any external data fed to the agent, cross-checking facts against trusted sources, and periodically resetting or auditing the agent's internal state. For example, a system could clear an agent's "short-term memory" after each sensitive transaction, or require digital signatures on any new knowledge items. Protect AI Agents With Layered Security Controls Identity and access control for agents is equally critical. Agents act as non-human service identities, so if an attacker steals an agent's credentials, they essentially hijack its privileges. Recorded Future warns that "compromised credentials, SSO platforms, or agent identities could enable large-scale... data exfiltration". In practice, a stolen token could let an attacker quietly siphon data or trigger commands anywhere the agent has access. To counter this, enterprises should issue each agent a unique short-lived token and restrict its scope strictly. Here, each API call by the agent uses a fresh, scoped token. If the token is leaked or abused, its very short life and limited permissions contain the damage. In practice, agent tokens should be rotated frequently, and every action should be logged under the agent's identity. If an agent suddenly tries to access an unexpected endpoint, automated policies should block or flag the request. In essence, treat agents like privileged users with their own IAM lifecycle by implementing least-privilege roles, multi-factor approvals for high-value operations, and full auditing of their activities. Multi-agent workflows introduce additional complexity. Agents often invoke other tools or orchestrate chains of sub-agents. In such pipelines, a compromise anywhere can cascade. For example, if Agent A trusts a data input or command from Agent B, and B has been misled or maliciously tampered with, A may unknowingly act on bad instructions. To mitigate this, every handoff between agents or tools should be authenticated and checked. Enforce endpoint authentication and message signing on each channel between agents, and apply authorization checks at every step. Segmentation and strong encryption on inter-agent communications can prevent a breach in one component from jumping to others. Monitor AI Agents for Anomalies and Unauthorized Actions At runtime, anomaly detection and monitoring provide a final safety net. Agents in production should exhibit well-defined baselines of behavior. An agent that usually looks up customer records, for instance, should not suddenly be streaming large volumes of payroll data. Security telemetry that logs every prompt, response, and tool invocation lets defenders spot when an agent deviates from its norm. Modern SIEM and AIOps platforms can ingest these logs and flag unusual patterns (for example, spikes in outbound data or unexpected API calls). By correlating agent activity with traditional logs and threat intelligence, teams can detect and contain a misbehaving agent before it causes systemic damage. In summary, securing enterprise applications in the agent era means integrating AI-specific defenses throughout the stack. Zero-Trust principles apply fully where we treat each agent call as untrusted until verified, grant agents only minimal permissions, and require human approval for any high-impact decision. Defense-in-depth remains essential as it sanitizes every input, isolates AI subsystems from sensitive resources, and monitors all outputs continuously. Industry standards are beginning to catch up. For example, NIST's new AI Risk Management Framework and the Cloud Security Alliance's guidelines explicitly recommend continuous threat modeling, red teaming of AI, and traceability for data and models. Ultimately, the agentic AI era raises the security stakes from theory into daily practice. Organizations that build AI-aware threat modeling, least-privilege IAM, prompt filtering, and anomaly monitoring into their DevSecOps pipelines will be best equipped to embrace AI agents safely. By doing the hard work now by integrating model security into the software lifecycle, enterprises can unlock the productivity of agents while keeping adversaries at bay.