Early Gains and Challenges in Enterprise AI Adoption: From ROI to Edge Computing

Snowflake customers eke out early gains from Gen AI applications

Much of the debate over artificial intelligence (AI) in the enterprise, especially the generative type of AI (Gen AI), is focused on statistics, such as the number of projects in development or the projected cost savings of automation, and the benefits are still very much hypothetical. To cut through some of the stats, and the theory, it can be useful to listen to Gen AI users, as I did during a dinner hosted in New York last week by data warehouse vendor Snowflake. Also: 5 tips for choosing the right AI model for your business The company invited prominent customers to speak about their experiences putting AI applications into production. The overall impression was that there are meaningful use cases for AI, including document search, which can start delivering benefits within six months or less of implementation. The conversations were anecdotal, and Snowflake is interested in promoting best-case scenarios from its customers to promote its cloud data warehouse services. Nonetheless, with that caveat in mind, the thoughtful comments by both customers suggest that companies create value by taking the plunge into AI with even very simple use cases, after only days, weeks, or months in production. Also: Today's AI ecosystem is unsustainable for most everyone but Nvidia, warns top scholar Thomas Bodenski, chief operating officer and head of analytics for TS Imagine, which sells a cloud-based securities trading platform, described how it would traditionally take 4,000 "man hours" of labor at his company to have people read through emails for crucial, actionable events. "I get mail, every year, 100,000 times, somebody that we buy data from, telling me that in three months, we're making a change," explained Bodenski. "If I'm not ready for this, there's 500 clients that will be down," meaning they will be unable to trade, he said. "So, it's very critical that you read every single email that comes in." Bodenski continued: "That email comes in, I have to classify it, I have to understand it, I have to delegate it to the right people, across different departments, to action it on it -- that task costs me 4,000 hours a year." That task has traditionally been the role of "a team around the globe" he oversees. There are at least two and a half "full-time equivalent" individuals, he said, "and they have to be, like, smart people." Also: Microsoft introduces 10 AI agents for sales, finance, supply chain in Dynamics 365 Bodenski said: "Now, I'm doing it at 3% of the cost of the people that would do that work," using a generative AI application. "Just do the math," said Bodenski. "You take the average salary and then calculate how much you spend on Snowflake, and that's just 3% of that cost." This email-reading program was the first app that TS Imagine built with Snowflake's help, said Bodenski. It was built using Meta Platforms's open-source Llama large language models and Snowflake's open-source alternative, Arctic. Those large language models employ retrieval augmented generation (RAG), where the model taps into an external database. The app "took six months of trial and error learning," said Bodenski. That process began before TS Imagine had a relationship with Snowflake. Then Snowflake introduced Cortex AI," the managed LLM inference service run by Snowflake, "we migrated the entire RAG pipeline over in four days, and now we are able to conceptualize a different story." Also: Snowflake says its new LLM outperforms Meta's Llama 3 on half the training The Cortex AI service allowed Bodenski to classify incoming customer emails for sensitivity, urgency, and other parameters, something that would not have been possible before "because I don't, like, you know, read all 5,000 customer emails coming in every month," he said. With classification, Bodenski said the result is that "I detect the brushfire before it even becomes a fire," meaning a customer mishap. "It is reliable, I have no problems, I don't miss a single email." TS Imagine now has six apps up and running using Gen AI, said Bodenski, "and I'm going to do much more. AI is going to continue to build our brains," he said: "It works." Snowflake customer S&P Global Market Intelligence had a similar experience, according to Daniel Sanberg, the head of "quantamental research" for the firm, who was also a guest at the dinner. Also: The journey to fully autonomous AI agents and the venture capitalists funding them Sanberg's company implemented an in-house application called Spark Assist on top of its Microsoft Office apps. Now, the firm can auto-generate email summaries. "The Gen AI is smart enough to know which ones are most relevant that need my immediate attention versus those that maybe need to be de-prioritized, and I just say [to the AI model], 'Go ahead and write a response to these,' and then I spot-check them." The app is used by 14,000 employees at S&P Global, said Sanberg. "I don't think I could go back," he said, referring to the old way of trying to sort and sift email manually. But does the return on investment of such apps justify the cost of building apps and the cost of inference? "I would say, finger to the wind, yes," said Sanberg, although he added: "I think we're still sizing a lot of these things." Sanberg continued: "The question is, in aggregate, what does that payoff look like? That's TBD. But in individual instances, sure; things that used to take days or longer to compile can now be done within a day [using Gen AI]." He compared Gen AI to the early days of the internet when dial-up speeds hampered the payoff for the average user. Also: Bank of America survey predicts massive AI lift to corporate profits "If we're sitting here and have to wait 15 minutes to log on" to the internet via dial-up modem, "is it really worth it?" Sanberg remarked. "But, it's not where we are now," he said. "It's where we'll be in five years; I think a lot of this stuff will get sorted." Snowflake's head of AI, Baris Gultekin, was also at the dinner and said Gen AI can already offer better economics to automate some tasks. Also: Asana launches a no-code tool for designing AI agents - aka your new 'teammates' "Cortex Analyst is this product that allows someone to ask a question, to get answers from the data, instantly," he explained. "The current pricing for 1,000 questions is $200, so, 20¢ a question. This is a question that otherwise would have to be answered by an [human] analyst. They would write the SQL [database] query for every single one of them. So, imagine 1,000 SQL queries. Each one takes, let's say, 10 minutes. You can see the ROI: 10 minutes a question, 1,000 questions, versus $200." Of course, twenty cents here and twenty cents there can add up, said Chris Child, vice president of worldwide sales engineering for Snowflake, a guest at the dinner. The key thing, he said, is for enterprises to be able to forecast how costs will add up as inferencing begins. "In most cases, people have set aside a budget," Child said. "They're thinking of grand things, and it's much more about, 'How do I understand how much is it going to cost me over a series of months, and how do I know when it's trending higher than that?'" Also: Gartner's 2025 tech trends show how your business needs to adapt - and fast His suggestion: "Try it, run it once, see, and then estimate what you're going to need to do it at scale." Child continued: "The cost of testing a hypothesis is high," versus, "If I'm going to spend $1,000 to run a first test case, it's still expensive, but it's dramatically cheaper" than using people to test the same hypothesis. When S&P Global put together an app using Snowflake for its clients, the tool aimed to sort through 12,000 historical quarterly financial filings issued by companies in the Russell 3000, the index of investible US companies, for 10 years across a total of 120,000 documents. "The first thing we did when we got on the platform was write a script that helped us calculate the cost before we pushed run, and we were able to do that," said Sanberg. "I like the consumption-based model," he said, referring to Snowflake's practice of billing customers for the total actual time used rather than a traditional software usage license, "because there's transparency in the pricing, because there's, in my opinion, fairness across the board." Also: There are many reasons why companies struggle to exploit generative AI, says Deloitte survey TS Imagine's Bodenski said flexibility in pricing of running inference in Cortex has worked for his needs. "I can run a process where I'm okay to wait three minutes for each prompt, but I can also run a process where it's not okay to wait three minutes, I want it in five seconds," he explained. "And I make the decision on the fly, just by increasing something from extra-small to medium," referring to the scale of compute. Bodenski said the app used by TS Imagine to hunt through emails showed its worth quickly. "We saw the impact, actually, four days after we designed it," he said, "because it surfaced those items that we needed to focus on, and it improves our customer service quality." The app has now been in production for four months. "It is very, very important for us," he said. "It elevates me to detect an item that I should be involved in, or my regional manager, my global head," said Bodenski. "It runs automated, it produces results, we're catching items" that might have taken weeks otherwise to receive a response in email, "and I didn't have to hire a single person or reallocate a single person to do that process."

Generative AI adoption sets the table for AI ROI - SiliconANGLE

Generative artificial intelligence enthusiasm has lately turned to artificial intelligence skepticism. Lack of clarity on tangible return on investment for mainstream businesses, a narrow list of early winners and relentless vendor marketing around AI has caused cynicism and media backlash. But the reality remains that we have entered a new era in technology innovation that has a high probability of transforming industries, public policy, company leadership and the related fortunes of individuals and organizations. Moreover, 97% of leading gen AI adopters report that they're achieving tangible benefits from their deployments. Adoption of gen AI is relentlessly on the rise and nearly two years in, is poised to begin throwing off enough value that it will heighten a mandate to apply AI to drive business results. As such, we believe that as we exit Q4 into 2025, the demand for AI solutions will continue to occupy the headspace of business technology pros and AI momentum will maintain its accelerated pace. In this Breaking Analysis, we highlight fresh data from the latest Enterprise Technology Research drill-down on generative AI to look at adoption rates, production use cases, ROI expectations, benefits realized and current spending patterns. Barely a day goes by where you don't see some type of negative article about AI ROI, references to Gartner's trough of disillusionment and the like. From the Wall Street Journal to the Register to trade publications and TV programs, the narrative is decidedly downbeat when it comes to AI ROI. But the last headline in the lower left is instructive. While AI ROI may be elusive for many organizations, enthusiasm that the bubble will continue persists. Today we unpack the most recent gen AI survey from ETR's latest drill-down. This is fresh data from nearly 1,800 information technology decision makers across industries and company sizes. It's weighted toward larger spendings in North America with a nice mix of C-level executives, middle managers and practitioners. The first set of data we'll look at shows that while there are still some holdouts, the vast majority of respondents (84%) are leaning into gen AI. Notice the steep decline in those firms that have not evaluated gen AI. I recall in the summer of last year being struck that one-third of the survey base was not evaluating gen AI. And the reasons they gave were that it was too risky or moving too fast. Well, that figure is down to 13% of respondents after a steep downward trajectory. A full 84% of respondents have clarity on at least one use case they're contemplating. This is relevant because organizations told us a year ago that they were inundated with use cases and had to prioritize and pare down the many ideas flowing in from lines of business. The point is today we're seeing much more clarity from organizations and that bodes well for ROI, which we'll explore later. Let's double-click on use cases and zoom in on those dominating the landscape. Many pundits have called 2023 the year of experimentation and that was largely true. You can see above on the leftmost bars - that dark blue bar is the July survey - and only 25% of the respondents weren't yet in production. So to review - from the previous chart, about 85% of customers are leaning in to gen AI and this data shows nearly 75% have at least one use case in production. In the red text we show which use cases are most popular. Text and data summarization at 31%, collaboration is 28%, sales and marketing content development and code generation next and so on -- with very few respondents saying they don't know. Now these use cases are pretty straightforward and some of the other higher net present value use cases are in that small "other" category in the low single digits. While these are not earth-shattering use cases, they're becoming widespread. Let's explore the ROI question. Does it matter that these examples are what we sometimes call "chatty," meaning they're ChatGPT-like? Well, from an ROI perspective, no, not in our view anyway. The chart above shows how the ROI expectations have pushed to the right - with now 21% saying the breakeven will be more than one year out, up from 13% in the April survey. So while here we see an indication of uncertainty with 21% (one year+) + 23% (not sure) = 44% saying more than a year or not sure. Nonetheless, a full 56% say they saw or expect to see ROI inside 12 months, which is encouraging, despite the fact that customers are being more cautious about ROI. That's the nature of these technology waves. When the surf's up like it is now, you grab the board and dive in. Because your competitors are out there stealing the best rides. As then-VMware Chief Executive Pat Gelsinger famously said on theCUBE in 2012, if you're not riding the waves, you'll end up as driftwood. Let's keep digging into the ROI question and look at how ITDMs see the benefits of deploying gen AI. Ninety-eight percent of the respondents in the previous chart answered the question posed in the chart above - i.e., what benefits are you seeing specifically from gen AI deployments? Seventy-seven percent said increased productivity and better efficiency. Thirty-nine percent cited better customers support - contact centers is a big initial use case. And you can see, the other benefits customers cite like better engagement with customers, product innovation, better data analysis, headcount savings and so on. Notably, 33% said cost savings on personnel, which is a big percentage of customers and we felt it was worth highlighting in the red. We don't know the degree of headcount savings, but the fact that a large proportion of customers cites that benefit directly is instructive. Note that only 3% of respondents said "none of the above," so clearly most customers deploying gen AI are seeing tangible benefits. Financial services is a bellwether industry. When you drill further into the data and look at a leading cohort like financial services you see even better results. Above we show 152 respondents in financial services and the percentage citing productivity improvements jumps to 84%, with 50% seeing improvements in customer support. Only 1% said "none of the above." The reason we feel this is important is that over the course of IT history, financial services has often been the leading adopter of the latest and greatest technologies. Now they were slower with the cloud, but that was a matter of both trust and cloud maturity -- whereas financial firms can exercise better control in this AI cycle, whether doing so in the cloud with virtual private clouds or working on-premises. It's important to ground this data in reality. Remember, ROI, technically, is a ratio of benefit over cost. Breakeven period refers to the time it takes to recoup your initial investment. Net present value or NPV is the discounted cash flow of an initiative over a time period and reflects the actual dollar value of an investment. In surveys like this, these concepts are often bundled into a single vague metrics called ROI, which is the case here. The point is when you look at an investment, it's important to evaluate the amount of money invested as we show below. From a strict ROI definition (benefit/cost), a lower denominator will increase the ROI percentage. It will generally be lower-risk and often result in smaller NPVs. ETR asked respondents to quantify their gen AI spend and you can see above that most firms are spending less than $100,000 annually. About half the survey spends less than $500,000. The reason this is relevant is because lowering the initial costs will naturally increase the percentage ROI. But this gives no indication of the timeframe to get to cash flow positive and no indication of size of benefit. As you can see above, about 30% spend more than $500,000 per annum and 23% spend north of $1 million annually, with nearly 10% spending more than eight figures this year. The point is firms are spending. We know that around 45% of customers are stealing from other budgets to fund gen AI. In financial services, around 20% of respondents are spending more than $5 million this year. The big spenders are getting really serious. For example, when we cut the data on the Global 2000, we see that 26% of this cohort are spending more than $5 million this year. And these big-spender firms might see a smaller ROI percentage but they'll likely see bigger NPVs. And they will set the pace for future investments -- especially as and if these investments start to throw of cash and enable reinvestment. The other thing to remember is that gen AI specifically and AI generally will become ubiquitous. The tech industry is embedding AI throughout the stack so as AI becomes less visible it will begin to drive new levels of productivity. It has been a while since we've seen a long sustained period of above average productivity growth. Above is a chart from the Bureau of Labor Statistics. The data shows the five-year moving average of annual productivity growth over six to seven decades. The 1950s and 1960s saw consumer spending become a dominant driver of productivity growth coming as the post-World War II industrial economy kicked in. But for the better part of the '70s and '80s, we saw generally below average productivity growth until the personal computer revolution kicked in and put a computer at everyone's fingertips. Then after the dot-com boom and bust, we saw productivity growth bottom at the start of the financial crisis of 2008/2009; and then a rebound to just above the post-WWII average. But that wave was largely false momentum driven by the recovery versus a major technological wave. The promise of gen AI specifically and AI overall is it will boost productivity growth by a meaningful amount. Erik Brynjolfsson said last year he'd be disappointed if we don't see a 3% to 4% improvement in productivity growth from AI, which would be noticeable on the above diagram. That's the hope for the technology industry to address many of the world's challenges including massive debt, climate change, poverty, overpopulation, terminal diseases and the like. Now our industry has made many promises in the past. Some have failed to deliver, but in the grand scheme of things, the technology industry has a pretty good track record of delivering societal value. In the near term, below we cite a few of the items we're watching as indicators of progress in the AI ROI discussion. Private data. Watch for the use of private data in training and tuning large language models and deploying gen AI specifically. We discuss this frequently, citing theCUBE Research Gen AI Power Law. That concept refers to the long tail of use cases and applications that will leverage smaller, domain-specific models. We're starting to see a lot more discussion around smaller models, which have hit the market recently, along with open-source models. We see proprietary data becoming a new form of competitive advantage as companies leverage their own data. RAG adoption. In the July drill-down, only 7% of the customers ETR surveyed were deploying retrieval-augmented generation. That was a surprisingly low figure. Now RAG maybe doesn't set the world on fire, but it does start to set the base. A lot of the experimentation that is being done is accomplished with RAG, and those systems provide learnings and will go into production in the near future to drive additional productivity. Spending levels on gen AI. We're also watching spending levels particularly watch the ROI denominator. When the denominator is small, you have big ROIs, but we're expecting bigger denominators which will lower the percent return but make the value bigger. It also might take longer to deliver break even, but higher NPVs could serve as a beacon of the potential for gen AI value in the future. If these initiatives throw off positive value/cash flow, it will trigger gain sharing that we talked about earlier. Use cases beyond ChatGPT. Let's think about use cases beyond those that are "chatty." Drug discovery, novel climate solutions. For example, maybe reducing carbon emissions is not the answer. Maybe the AI will help us figure out ways to take carbon out of the atmosphere. New energy solutions to power AI. Maybe let AI bloom and help figure out how to identify better energy solutions, cures for terminal diseases, solutions for world hunger and poverty. Also, expect new military technologies in the form of drones and other intelligent devices. Not to mention cybersecurity. Agentic AI. We're also watching the rise of agents as a productivity booster. Agents has become the hot new buzzword beyond single co-pilots. Single agents really aren't as interesting, but when you start to apply "swarms" of agents that can learn from reasoning traces of humans and connect to data from back end systems and be guided by top-down metrics of an organization, this is going to be a signal for real productivity boosts. We're talking here about true end-to-end automation and the emergence of what we sometimes call AI-native companies that dramatically change the way in which organizations operate with new workflows that require one-10th of the people to do today's work -- and the expectation that outcomes will be achieved much faster over a dramatically compressed elapsed time. What about it? Are you an optimist or a skeptic on AI ROI? What are you seeing in your organization and how do you think 2025 will play out?

Today's AI ecosystem is unsustainable for most everyone but Nvidia, warns top scholar

The economics of artificial intelligence are unsustainable for just about everyone other than GPU chip-maker Nvidia, and that poses a big problem for the new field's continued development, according to a noted AI scholar. Also: Gartner's 2025 tech trends show how your business needs to adapt - and fast "The ecosystem is incredibly unhealthy," said Kai-Fu Lee in a private discussion forum earlier this month. Lee was referring to the profit disparity between, on the one hand, makers of AI infrastructure, including Nvidia and Google, and, on the other hand, the application developers and companies that are supposed to use AI to reinvent their operations. Lee, who served as founding director of Microsoft Research Asia before working at Google and Apple, founded his current company, Sinovation Ventures, to fund startups such as 01.AI, which makes a generative AI search engine called BeaGo. Lee's remarks were made during the Collective[i] Forecast, an interactive discussion series organized by Collective[i], which bills itself as "an AI platform designed to optimize B2B sales." Today's AI ecosystem, according to Lee, consists of Nvidia, and, to a lesser extent, other chip makers such as Intel and Advanced Micro Devices. Collectively, the chip makers rake in $75 billion in annual chip sales from AI processing. "The infrastructure is making $10 billion, and apps, $5 billion," said Lee. "If we continue in this inverse pyramid, it's going to be a problem," he said. The "inverse pyramid" is Lee's phrase for describing the unprecedented reversal of classic tech industry economics. Traditionally, application makers make more money than the chip and system vendors that supply them. For example, Salesforce makes more money off of CRM applications than do Dell and Intel, which build the computers and chips, respectively, to run the CRM applications in the cloud. Also: Bank of America survey predicts massive AI lift to corporate profits Such healthy ecosystems, said Lee, "are developed so that apps become more successful, they bring more users, apps make more money, infrastructure improves, semiconductors improve, and goes on." That's how things played out not only in the cloud, said Lee, but also in mobile computing, where the fortunes of Apple and ARM have produced winners at the "top of the stack" such as Facebook's advertising business. Conversely, "When the apps aren't making money, the users aren't getting as much benefit, then you don't form the virtuous cycle." Returning to the present, Lee bemoaned the lopsided nature of Nvidia's marketplace. "We'd love for Nvidia to make more money, but they can't make more money than apps," he said, referring to AI apps. The development of the ecosystems of the cloud, personal computers, and mobile "are clearly not going to happen today" at the current rate of spending on Nvidia GPUs, said Lee. "The cost of inference has to get lower" for a healthy ecosystem to flourish, he said. "GPT-4o1 is wonderful, but it's very expensive." Lee came to the event with more than a warning, however, offering a "pragmatic" recommendation that he said could resolve the unfortunate economic reality. He recommended that companies build their own vertically integrated tech stack the way Apple did with the iPhone, in order to dramatically lower the cost of generative AI. Also: The journey to fully autonomous AI agents and the venture capitalists funding them Lee's striking assertion is that the most successful companies will be those that build most of the generative AI components -- including the chips -- themselves, rather than relying on Nvidia. He cited how Apple's Steve Jobs pushed his teams to build all the parts of the iPhone, rather than waiting for technology to come down in price. "We're inspired by the iPhone," said Lee of BeaGo's efforts. "Steve Jobs was daring and took a team of people from many disciplines -- from hardware to iOS to drivers to applications -- and decided, these things are coming together, but I can't wait until they're all industry-standard because by then, anybody can do it," explained Lee. The BeaGo app, said Lee, was not built on standard components such as OpenAI's GPT-4o1, or Meta Platforms's Llama 3. Rather, it was assembled as a collection of hardware and software developed in concert. "Through vertical integration, [we designed] special hardware that wouldn't work for necessarily other inference engines," explained Lee. For example, while a GPU chip is still used for prediction-making, it has been enhanced with more main memory, known as high-bandwidth memory (HBM), to optimize the caching of data. Also: Businesses still ready to invest in Gen AI, with risk management a top priority The software used for BeaGo is "not a generic model." Without disclosing technical details, Lee said the generative AI large language model is "not necessarily the best model, but it's the best model one could train, given the requirement for an inference engine that only works on this hardware, and excels at this hardware, and models that were trained given that it knows it would be inference on this hardware." Building the application -- including the hardware and the novel database to cache query results, has cost BeaGo and its backers $100 million, said Lee. "You have to go back to first principles, and say, 'We want to do super fast inference at a phenomenally lower costs, what approach should we take?' " Lee demonstrated how BeaGo can call up a single answer to a question in the blink of an eye. "Speed makes all the difference," he said, comparing it to Google's early days when the new search engine delivered results much faster than established engines such as Yahoo! Also: AI agents are the 'next frontier' and will change our working lives forever A standard foundation model AI such as Meta's Llama 3.01 405b, said Lee, "will not even come close to working out for this scenario." Not only is BeaGo able to achieve a greater speed of inference -- the time it takes to return a prediction in response to a search query -- but it's also dramatically cheaper, said Lee. Today's standard inference cost using a service such as OpenAI's GPT-4 is $4.40 per million tokens, noted Lee. That equates to 57 cents per query -- "still way too expensive, still 180 times more expensive than the cost of non-AI search," explained Lee. He was comparing the cost to Google's standard cost per query, which is estimated to be three-tenths of one cent per query. The cost for BeaGo to serve queries is "close to one cent per query," he said, "so, it's incredibly inexpensive." The example of BeaGo, argued Lee, shows "what needs to happen to catalyze the [AI] app ecosystem [is] not going to happen by just sitting here using the newest OpenAI API, but by someone who dares to go deep and do that vertical integration." Lee's dour overview of the present contrasts with his conviction that generative AI will enable a new ecosystem that is ultimately as fruitful as the PC, cloud, and mobile eras. Also: The best AI chatbots: ChatGPT, Copilot, and worthy alternatives "Over the next two years, all the apps will be re-written, and they will provide value for the end user," said Lee. "There will be apps that didn't exist before, devices that didn't exist before, business models that didn't exist before." Each step of that development, said Lee, "will lead to more usage, more users, richer data, richer interaction, more money to be made." Those users "will demand better models and they will bring more business opportunities," he said. "It took the mobile industry 10 years to build [a successful ecosystem]," he said. "It took the PC industry perhaps 20 years to build it; I think, with Gen AI, maybe, two years." Lee offered his thoughts on what the consumer and enterprise use cases will look like if generative AI plays out successfully. For consumers, he said, the smartphone model of today most likely will go away. "The app ecosystem is really just the first step because once we start communicating with devices by speech, then the phone really isn't the right thing anymore because we are wanting to be always listening, always on, and phones are not." Also: Think AI can solve all your business problems? Apple's new study shows otherwise As for app stores, said Lee, "they'll be gone because agents will directly do things that we want, and a lot of apps and e-commerce -- that will change a lot, but that's later." The path for enterprise use of generative AI is going to be much more difficult than the consumer use case, hypothesized Lee, because of factors such as the entrenched nature of the business groups inside companies, as well as the difficulty of identifying the areas that will truly reap a return on investment. "Enterprise will go slower," he said, "because CIOs are not necessarily fully aligned with, and not necessarily fully knowledgeable about, what Gen AI can do." Likewise, hooking up generative AI to data stored in ERP and CRM systems, said Lee, "is very, very tough." The "biggest blocker" of Gen AI implementation, said Lee, "is people who are used to doing things one way and aren't necessarily ready to embrace" new technological approaches. Also: AI agents are the 'next frontier' and will change our working lives forever Assuming those obstacles can be surmounted, said Lee, early projects in Gen AI, such as automating routine processes, are "good places to start, but I would also say, these are not the best points to create the most value. "Ultimately, for enterprises, I think Gen AI should become the core brain of the enterprise, not these somewhat peripheral things. For an energy company, what's core is drilling oil, right?" Lee offered. "For a financial institution, what's core is making money." What should result, he said, is "a smaller, leaner group of leaders who are not just hiring people to solve problems, but delegating to smart enterprise AI for particular functions -- that's when this will make the biggest deal." Also: 6 ways to write better ChatGPT prompts - and get the results you want faster "What's really core is not just to save money," said Lee, "but to make money, and not just any money, but to make money in the core, strategic part of the company's business."

Operationalizing AI at the edge -- and far edge -- is the next AI battleground

As more organizations charge into the AI and machine learning fray, technology and operations leaders are keeping one eye on their competition, and the other on how their own AI workloads are impacting their infrastructure needs now -- plus worrying what the near future will demand in terms of evolving technology and sophisticated infrastructure. There's plenty to worry about. The number, size and complexity of AI workloads is skyrocketing, which is alarming news for data centers, which are already pressed to the limit trying to power, cool and find space for the high-octane infrastructure required to stay just ahead of the ravening horde. On the chipset side, innovation is a blur, giving companies behind large foundational and frontier models the power they need to keep pushing those models forward, and making them ever larger and more computationally complex. The hardware that started the AI revolution is arguably antiquated now for the current model sets, and refresh iterations are getting shorter all the time. Where an organization could expect to keep infrastructure for five to seven years, they're finding that number shrunk to months-long lifecycles instead. Space issues magnify, as companies scramble to make room for migrations and growth. There are some clear directional signals, says Mike Menke, field CTO at AHEAD, and key among them is the push towards edge computing across industries. "We're talking about very power-hungry, very large systems," Menke says. "While we're going to alleviate some of the data center constraints, we'll continue to push the boundaries of what we can do at the edge and the far edge as we try to get the computational part of it closer to the end consumer or user. The edge is where inference is ultimately going to end up." Edge adoption across industries Edge computing is used widely in a broad array of asset-intensive industries, like manufacturing, utilities, retail, healthcare and transportation and logistics, where companies are pairing live streams of data from an ecosystem of edge sensors and IoT devices with AI to make real-time decisions. Adoption is growing, too -- Gartner predicts that by 2025, 75% of enterprise-generated data will be created and processed outside a traditional centralized data center or the cloud, and a big chunk of that growth comes with new use cases across all industries. A growing demand for machine learning, AI and low-latency data analytics will continue to accelerate the demand for edge computing solutions. Edge AI can collect and analyze data and deliver insights that drive efficiency, reduce safety and security risks, and help organizations deliver more value to customers. It's also applied in use cases that require lower latency and data gravity (or the need to reduce bandwidth costs by processing data where it's generated), and where greater resilience is needed during disconnection. That includes not just physical security systems using computer vision to detect and report safety and security incidents, but data-driven clinical decision support (CDS) solutions for healthcare providers. And as the demand for real-time analytics continues to grow, organizations will require more data processing and inferencing at the edge, and ongoing rapid innovation in distributed cloud solutions, device orchestration and management software, edge application platforms and other technologies will keep driving edge adoption. Edge and far edge, now feasible at scale Implementing edge solutions has been a challenge until now, for a variety of reasons says Bill Conrades, senior director of engineering at AHEAD. "Edge AI isn't just the hardware and the software, it's also the sensors involved in generating the live stream of data," Conrades says. "The vendor ecosystem is extremely fragmented and most IT departments aren't set up to integrate and maintain their edge devices at scale. You need supply chain transparency, asset management, integration and configuration services, plus life-cycle management transparency, so you know when to refresh your licenses and support contracts." And as AI and generative AI use cases multiply, organizations have run up into all-too-common roadblocks, including the need for major engineering, data science and IT infrastructure investments that require a serious number of tools and a whole lot of expertise that's fairly thin on the ground these days, no matter the industry. Some significant factors have changed the game, making far edge, in particular, more feasible. Edge management and orchestration platforms (EMO) allow zero-touch provisioning and upgrades in the field, which is a huge deal in industries that work with far-flung and difficult-to-access locations, like oil rigs, solar farms and rail lines that span the country. Most of those deployments have hundreds, if not thousands, of edge devices inferencing out at the far edge, and far from the core data center where the training takes place -- and where the IT staff lives. With an (EMO) solution, the IT team can monitor sensors and the status of the system, deploy updates to the core AI model itself after it's been fine-tuned with additional new data sets or parameters and update the hardware's firmware if necessary. "A lot of the customers see the benefit in managing their edge applications just like they manage their cloud applications," Conrades says. "They want to manage their systems in the same way. A lot of these edge orchestration solutions support containers or virtual machines. They also support passing through the accelerator to the application with very little latency." Getting ready for the future Organizations need to prepare to meet both the current state of AI as well as its future demands, and they're both endlessly moving targets. "A lot of organizations are used to having finite timelines on investments, projects taking a certain time," Menke says. "Today it's understanding that this is early days, and innovation is happening very quickly. You may be working on a problem and you wake up tomorrow and somebody on the other side of the world solved it while you were asleep. Being willing to grab that, bring that into your development process and take advantage of it is more important now than it's ever been." Conrades points out that AI isn't new at all -- generative AI has simply brought traditional AI, including machine learning and deep learning more clearly into the spotlight, prompting more organizations to discover exciting new use cases. That means the wellspring of professional expertise available is deep. "For instance, pull in a long-standing ISV that has a developed solution for your use case, operationalize AI and start solving today. Partner with an organization like AHEAD that can bring resources to the table to help you in your AI journey with pre-trained models, fine-tuning expertise, building blocks of hardware and a program that makes it easy and far faster," he says. "I guarantee your competition is already picking up and operationalizing AI in some way shape or form."