Indian Enterprises Struggle to Scale AI as 67% Report Infrastructure Unready for Production Workloads

Infrastructure Bottleneck Threatens Enterprise AI Ambitions

Indian enterprises are investing heavily in artificial intelligence, but a critical gap between AI adoption in enterprises and infrastructure readiness threatens to derail production deployments. Dell Technologies' Modern Enterprise Readiness Study reveals that 67% of Indian organisations believe their current data centre environments are not ready to support AI-driven workflows at production scale1

1

. The challenges in scaling AI agents extend beyond model capabilities to fundamental questions about whether existing digital infrastructure can reliably support autonomous systems that must retrieve information from enterprise databases, connect with business applications, and execute tasks across fragmented technology stacks.

The infrastructure bottleneck manifests most acutely when AI agents attempt to operate across legacy applications, distributed data environments, and architectures never designed for software capable of independent decision-making. "The biggest hurdle in moving an AI agent from pilot to production is rarely the model itself. It's the reality of integrating with fragmented enterprise data, legacy applications, security controls, and business processes that have evolved over decades," explains Vishal Kappanna, Engineering Director at Tesco Technology1

1

. This reality check comes as Gartner predicts over 40% of agentic AI projects will be cancelled by the end of 2027 due to escalating costs, unclear business value, and inadequate risk controls5

5

.

Legacy Systems and Integration Complexity Block Scaling AI

Replacing entire technology stacks remains impractical for most Indian enterprises, forcing organisations to wrap and modernise legacy systems through application programming interfaces, middleware, and orchestration platforms. However, connecting an agent to an application does not automatically ensure reliable workflows. An AI agent processing a customer order must retrieve details from CRM and ERP integration systems, verify inventory against enterprise resource planning platforms, and initiate subsequent workflow steps—a process that fails when applications cannot exchange information reliably or return conflicting records1

1

.

"Most enterprises are not replacing legacy systems to enable AI agents. They're wrapping and modernising them through orchestration platforms, getting data out through events and APIs, and gating access behind governance layers and agent identities—modernising the seams, not the systems," notes Kappanna1

1

. This approach allows organisations to retain existing applications while exposing necessary capabilities, but technology debt accumulated over decades creates persistent constraints. Connecting older systems to orchestration layers does not resolve underlying architectural limitations that prevent production-scale AI deployment.

The gap between pilot demonstrations and production reliability exposes a fundamental truth about Enterprise AI implementation. "A pilot demonstrates that an agent can perform a task. Production requires proving that the enterprise can trust the agent to perform that task repeatedly, securely, and at scale," explains Vijay Yellapragada, Consulting AI and Data and US Commercial Leader at EY GDS1

1

. Accounting for system failures, process exceptions, and transaction volumes that never emerge during controlled experiments becomes as critical as the AI agents themselves.

Data Fragmentation Undermines AI Agent Performance

Even when autonomous AI systems connect successfully to enterprise applications, their performance depends entirely on data quality and accessibility across distributed environments. Large organisations store data across on-premises infrastructure, cloud platforms, and distributed repositories where records are frequently duplicated, outdated, or inconsistent across applications. "The biggest challenge is often not the AI model itself, but the data behind it. Most organizations have data spread across on-premises environments, cloud platforms, and edge locations, many of which were never designed to support AI-driven workflows at scale," says Premalakshmi PR, Managing Director and Area VP, India and SAARC at NetApp1

1

.

This data fragmentation creates operational bottlenecks that prevent AI agents from identifying which information to use when executing tasks. The data architecture challenge compounds when agents must operate across multiple systems simultaneously, each with different data formats, access permissions, and update frequencies. Making data discoverable without moving or duplicating it requires sophisticated platforms, but even then, organisations must address the fundamental issue of data consistency across fragmented technology environments.

Token Consumption Does Not Equal Monetizing AI Returns

Access to advanced AI capabilities does not automatically translate into productivity gains or profitability without proper infrastructure and process redesign. "Just because you consume tokens doesn't automatically mean there is a bank and you can start having a teller to withdraw money," observes Satish Viswanathan, former global data and AI lead for travel at Accenture2

2

. Companies treating token consumption as a measure of AI capability are discovering that consuming tokens without checks costs money without necessarily generating revenue.

Viswanathan identifies several factors determining how effectively organisations can use AI: model capability, enterprise data architecture and availability, system integration and infrastructure, human processes and organisational structures, value economics, and governance and regulation. Every enterprise carries its own "technology debt, people debt, process debt" and organisational-structure debt that constrains how much of a model's intelligence can actually be deployed productively2

2

. Even access to artificial general intelligence would not make every enterprise equally capable because companies cannot assume identical technology will produce uniform results across different organisational contexts.

The ROI of AI remains elusive for many organisations as competitive pressure drives adoption without clear monetization pathways. "We've already seen very, very smart pausing," Viswanathan notes, describing how enterprises are containing spending rather than pursuing full-throttle investment as chief executives face pressure to demonstrate returns from AI expenditure2

2

. Companies must determine specific use cases, areas of intervention, and revenue generation strategies before scaling investments.

Productivity Gains Require Organisational Transformation

Even when AI accelerates task completion, time saved does not automatically convert into financial benefits. An employee saving an hour through AI assistance might redirect that time to non-work activities rather than additional productive tasks. "We are not in this factory model where the moment we go at 9 am when you punch your card, you're invariably working," Viswanathan observes2

2

. Knowledge workers do not necessarily spend every hour on assigned tasks, meaning time freed by AI may not automatically redirect to other work unless organisations actively encourage productive redeployment through new responsibilities or skill development.

Technology services companies face additional challenges when AI enables faster work completion. Clients may expect efficiency gains reflected in contract prices, preventing providers from retaining all savings. The pressure to improve efficiency also raises workforce development concerns, as reducing entry-level hiring because AI can perform routine tasks creates long-term problems. "If you restrict people joining at the bottom of the pyramid, it's a disaster in the long term," Viswanathan warns2

2

. Employees build institutional knowledge through years of project work, decision observation, and learning from failures—experience that prepares them for senior responsibilities.

Accountability in AI Remains Critical for Production Deployment

Scaling autonomous AI within Indian enterprises remains constrained by trust concerns as businesses hesitate to grant AI agents authority over critical processes. "India has largely solved for access to AI. However, the infrastructure bottleneck is the confidence in delivering AI in a production environment, at scale," explains Rafi Katanasho, Asia Pacific CTO and VP of solution sales at Dynatrace3

3

. Autonomous AI systems can fail silently, delivering poor business outcomes without triggering system alerts, a particularly dangerous scenario when agents access systems, use tools, and execute tasks on behalf of employees.

Deloitte India's State of AI in the Enterprise report released in March 2026 found 40% of Indian respondents reported significant or full AI usage, compared with 28% globally, with strongest at-scale deployment in product development, strategy and operations, and marketing and sales3

3

. However, deploying AI differs fundamentally from granting agents authority to make decisions and act without human oversight. Many companies adopt a "bolt-on AI" approach, automating smaller process components instead of redesigning entire workflows around autonomous systems.

Indian banks experimenting with AI agents for know-your-customer processes and credit validation remain cautious about extending automation to complete business processes due to significant error consequences. Even 99.9% accuracy creates meaningful challenges when applied across large transaction volumes in India's digital payments ecosystem. "All of a sudden, you're not getting big errors, you're just getting bad business results," Katanasho observes3

3

. The challenge lies in identifying when agents behave unexpectedly rather than simply checking whether underlying applications run correctly.

Human Oversight and Regulatory Compliance Shape AI Deployment

"Autonomy requires authority. But authority without the truth is very dangerous," Katanasho emphasises3

3

. Companies need independent systems to observe all agent interactions end-to-end and hold outcomes accountable, not just individual agents. This requirement becomes critical in banking, fintech, and healthcare sectors where errors in automated processes affect sensitive data, service availability, and overall business operations.

RBI's FREE-AI framework emphasises accountability in AI through its focus on Safety and Resilience, making clear that AI agents should never have more authority than the employees they represent. "AI can be the co-pilot, but a human must remain the captain," states Pankaj Goel, CTO at LeadSquared4

4

. Regulatory compliance for AI demands that "the algorithm decided" cannot serve as an acceptable answer to regulators or customers. Every AI workflow requires a named human owner, an immutable audit trail, and human-in-the-loop oversight for high-stakes decisions.

The question for boards and chief financial officers increasingly centres on whether AI investment delivers measurable returns as companies redirect money from traditional technology investments toward AI. "There's a lot of tokens being consumed here. But what is the outcome? What are the business outcomes from that investment going on?," Katanasho asks3

3

. While employees may feel more productive using AI tools, the larger opportunity lies in redesigning entire workflows around AI agents—process reengineering through AI rather than adding AI to existing systems.

Architecture Determines Agent Reliability More Than Model Choice

The evolution from generative AI to agentic AI marks a fundamental architectural shift, moving software design from writing control flow logic to defining system intent. "My rough estimate is that model choice explains perhaps a fifth of how well an agent performs. The rest sits in the engineering decisions around it," says Ramakant Yadav, Co-Founder at Scalar Field5

5

. As models assume control over execution sequences, traditional development assumptions around determinism, exhaustive testing, and static exception handling no longer hold.

Reliability, not capability, emerges as the primary bottleneck. Models can already plan better than most deployments suggest, but consistency breaks across long chains of steps due to compounding errors. If every step in a chain maintains 95% reliability, twenty steps leave overall reliability at just 36%5

5

. Nothing in that chain appears broken, each individual decision remains defensible, yet the system still fails. Planning failures prove worse than generation failures because they surface late—agents pick reasonable-looking approaches early, execute multiple subsequent steps competently, and deliver confidently wrong results with errors buried under correct-looking work.

Building reliable autonomous systems requires separating the reasoning layer from deterministic execution—letting models handle high-level planning and decision-making while dedicated software components execute calculations, maintain state, and manage resource boundaries. Benchmark work in finance shows leading models scoring above 95% on simple lookups while collapsing to near zero on multivariate computation5

5

. Nearly every agent failure stems from architectural rather than model limitations.

Infrastructure Requirements for Production-Scale AI Deployment

Scaling agentic AI projects safely across enterprise environments relies on robust infrastructure rather than raw model capabilities. Four critical elements determine success: state and memory persistence across sessions transforms demos into dependable systems; isolation ensures every agent operates within defined boundaries to contain operational blast radius; observability provides step-level visibility to identify where multi-step processes fail; and resource controls implement hard cost limits and emergency kill switches essential for production deployment5

5

.

Gartner expects 33% of enterprise software applications to include agentic AI by 2028, against under 1% in 2024—a transformation requiring fundamental rewrites rather than upgrades5

5

. Enterprises must scale AI with the same discipline applied to building customer trust, understanding their data architecture, making consent machine-readable, enforcing least-privilege access, retaining human oversight for critical actions, and maintaining audit trails they would be comfortable showing customers. The winners will not necessarily be those adopting AI fastest, but those scaling it while giving customers confidence in how their data is used and how decisions are made. Watch for enterprises that successfully separate AI reasoning from deterministic execution, implement comprehensive observability across agent interactions, and establish clear accountability frameworks—these organisations will capture the larger benefits of AI-native process redesign rather than bolt-on automation.