The Evolution of AI: Data Labeling Challenges and the Rise of Not-Large Language Models

The hidden reason AI costs are soaring -- and it's not because Nvidia chips are more expensive

But companies training AI models, or fine-tuning existing models to improve performance on specific tasks, also struggle with another often overlooked and rising cost: Data labeling. This is a painstaking process in which generative AI models are trained with data that is affixed with tags so that the model can recognize and interpret patterns. Data labeling has long been used to develop AI models for self-driving cars, for example. A camera captures images of pedestrians, street signs, cars, and traffic lights and human annotators label the images with words like "pedestrian," "truck," or "stop sign." The labor-intensive process has also raised ethics concerns. After releasing ChatGPT in 2022, OpenAI was widely criticized for outsourcing the data labeling work that helped make the chatbot less toxic to Kenyans earning less than $2 hourly. Today's generic large language models (LLMs) go through an exercise related to data labeling called Reinforcement Learning Human Feedback, in which humans provide qualitative feedback or rankings on what the model produces. That is one significant source of rising costs, as is the effort involved in labeling private data that companies want to incorporate into their AI models, such as customer information or internal corporate data. In addition, labeling highly technical, expert-level data in fields like legal, finance, and healthcare is driving up expenses. That's because some companies are hiring high-cost doctors, lawyers, PhDs, and scientists to label certain data or outsourcing the work to third-party companies such as Scale AI, which recently secured a jaw-dropping $1 billion in funding as its CEO predicted strong revenue growth by year-end. "You now need a lawyer to label stuff, [which is] a crazy use of legal hours," said William Falcon, CEO of AI development platform Lightning AI. "Anything high stakes" requires expert-level labeling, he explained. "A chat with a 'virtual BFF is not high stakes, providing legal advice is." Alex Ratner, CEO of data labeling startup Snorkel AI, says corporate customers can spend millions of dollars on data labeling and other data tasks, which can eat up 80% of their time and AI budget. Over time, data must also be relabeled to remain up to date, he added. Matt Shumer, CEO and cofounder of AI assistant startup Otherside AI, agreed that fine tuning LLMs has gotten expensive. "Over the past couple of years, we've gone from middle-school-level data being okay to needing high school, college, and now expert," he said. "That obviously doesn't come cheap." That can create budget woes for tech startups building in important areas like healthcare. Neal Shah, CEO of CareYaYa, a platform for elder caregivers, says his company received a grant from Johns Hopkins University to build "the world's first AI caregiver trainer for dementia patients," but that data labeling costs are "eating us alive." The cost, he said, has skyrocketed 40% over the past year because of the specialized information needed from gerontologists, dementia experts, and veteran caregivers. He's working to reduce those costs by enlisting healthcare students and college professors to do the labeling. Bob Rogers, CEO of Oii.ai, a data science company specializing in supply chain modeling, said he has seen data labeling projects that cost millions. Platforms like BeeKeeper AI, he said, can help lower costs by letting multiple companies share experts, data, and algorithms without exposing their private data to the others. Kjell Carlsson, head of AI strategy at Domino Data Lab, added that some companies are lowering costs by using "synthetic" data -- or data generated by the AI itself -- to at least partially automate data collection and labeling. In some cases, models can fully automate any data labeling. For example, biopharma companies are training generative AI models for developing synthetic proteins for conditions like colo-rectal cancer, diabetes, and heart disease. The companies automatically conduct experiments based on the outputs of generative AI models, which provide new training data that comes with labels. The bottom line, however, is that data labeling may be costly and time-intensive, but well-worth it. "Data labeling's a beast," said CareYaYa's Shah. "But the potential payoff is massive." DeepMind military protest. Nearly 200 DeepMind staffers want Google's AI unit to stop working with the military, Time reports. A letter to management reportedly says Google's Cloud business is breaking the company's rules by selling AI to militaries that are at war -- no names are named, but there are links to reports on Google's dealings with the Israeli military and (allegedly) Israeli weapons firms. Google claims only Israeli government ministries are using its cloud services, with no "military workloads relevant to weapons or intelligence services." China's Amazon route. Reuters reports that state-linked entities in China have been using Amazon's cloud services to access the kind of advanced chips and AI that U.S. export controls aim to hold back from China. The U.S. rules ban exports and transfers of advanced chips and AI software to Chinese entities, but access through the cloud is allowed. Amazon Web Services says it's not breaking any rules. Cruise + Uber. GM's Cruise robo-taxi unit, which is trying to get back on track after serious setbacks, has struck a deal with Uber to offer self-driving services in an unspecified U.S. city, the Financial Times reports. Uber already has a similar arrangement with Alphabet's Waymo for robo-taxi services in Phoenix. That said, Cruise isn't offering autonomous rides right now -- it's still testing its cars with human drivers after a long pause that followed an incident in San Francisco in which a pedestrian was dragged underneath one of its cars. "You can find a few interesting use cases, but broadly, it seems like there's a lot of caution around this...Particularly around bigger companies that have complex permissions around their SharePoint or their Office 365 or things like that, where the Copilots are basically aggressively summarizing information that maybe people technically have access to but shouldn't have access to." -- Securiti chief data officer Jack Berkowitz tells The Register that half the peers he's polled have paused their rollouts of Microsoft's Copilot, an AI assistant that he alleges is accessing internal corporate data that it shouldn't. AI makes self-driving cars possible. So why is the industry keeping its distance?, by Sage Lazzaro Alibaba is upgrading its Hong Kong listing to primary, and that could potentially unlock billions in new investment, by Lionel Lim The stranded Boeing Starliner astronauts planned to hitch a ride home with SpaceX, but their spacesuits aren't compatible with Elon Musk's spacecraft, by Marco Quiroz-Gutierrez A California woman outsmarted two alleged mail thieves by sending herself an AirTag, by the Associated Press I sold a $1.4B big-data startup to IBM -- then founded a nature sanctuary. Here are the dangers of AI energy consumption, by Chris Gladwin (Commentary) Jelly Pong. Scientists have managed to make a "soft and squidgy water-rich gel" learn how to play the vintage video game Pong, the Guardian reports. What's more, the hydrogel actually gets better at the game with time as it has memory, though it is not sentient, the U.K. researchers said. However, the jelly-like material isn't as good a Pong player as another system that was shown off a couple years ago, based on a bunch of neurons in a dish -- satisfyingly, that system was named DishBrain.

Beyond OpenAI: The rise of not-too-large language models - SiliconANGLE

A flurry of new artificial intelligence models this week illustrated what's coming next in AI: smaller language models targeted at vertical industries and functions. Both Nvidia and Microsoft debuted smaller large language models too. Also supporting the notion of more customized models -- call them VLMs -- OpenAI made its GPT-4o fine-tuning generally available. As much as LLMs have captured much of the attention, these smaller, more controlled models look appealing to enterprises concerned about data governance and privacy, not to mention efficiency. Indeed, Chinese startups are heading in the same direction, partly to save energy and partly to avoid the need for the most advanced Nvidia graphics processing units to which they don't have access under export controls. That said, it looks like many Chinese companies are getting access to that high-end computing power through cloud providers such as Amazon Web Services. Advanced Micro Devices CEO Lisa Su doubled down this week on her quest to slice off a chunk of Nvidia's lucrative GPU market, as it acquired AI infrastructure provider ZT Systems. Infrastructure observability firms are having a moment. Not too long after Cisco Systems closed its acquisition of Splunk, others continue to reap the rewards, including Datadog turning in an upside quarter earlier this month. This past week, Grafana Labs raised a boatload at a $6 billion valuation. Snowflake shares dropped almost 15% Thursday after a disappointing revenue outlook as well as concerns about profitability. But everyone else had pretty positive earnings reports, including Palo Alto Networks, Workday, Synopsys, Zoom and Zuora. Autonomy founder Mike Lynch sadly died at sea off Sicily with several others, celebrating just a couple months after winning his long-running HP court case. Oddly, co-defendant Stephen Chamberlain was hit by a car and died earlier this week. Next week SiliconANGLE, theCUBE and theCUBE Research analysts will be at VMware Explore Monday through Wednesday to suss out what's happening with the virtualization and cloud pioneer under new owner Broadcom. Also next week: earnings reports from more bellwethers such as Nvidia, Salesforce, CrowdStrike, Dell, NetApp, Pure Storage, HP, MongoDB, HashiCorp and more. SiliconANGLE and theCUBE Research analysts John Furrier and Dave Vellante discuss this and other news in more detail on this week's theCUBE Pod, out later today on YouTube. And don't miss Vellante's weekly deep dive, Breaking Analysis, this weekend. Here's the big news of the week from SiliconANGLE and beyond: China finds a cloud workaround for high-end AI: Report: Chinese organizations use public cloud to access restricted AI chips More attention on AI training data: An AI holdout: Procreate says it won't ever use generative AI in its creative products OpenAI agrees content licensing deal with Condé Nast to feed SearchGPT and ChatGPT Opkey reels in $47M to automate ERP change testing with AI A key for agentic AI: AI payment processing startup Skyfire launches $8.5M in funding BeyondMath raises $8M to transform engineering and design with AI trained on world's knowledge of physics Piramidal raises $6M to advance AI brainwave analysis and improve diagnoses of neurological conditions Agribusiness AI startup Ceres Imaging rebrands as Ceres AI after closing on late-stage funding Nvidia, Microsoft release new small language models Juniper Networks rolls out AI networking blueprint to accelerate deployments OpenAI makes fine-tuning for GPT-4o customization generally available AI21 Labs' updated hybrid SSM-Transformer model Jamba gets longest context window yet Nvidia debuts StormCast generative AI model for forecasting mesoscale weather events Waymo debuts sixth-generation Driver autonomous driving platform Salesforce's newest AI agents help to filter out sales prospects and train salespeople Onehouse's vector embeddings support aims to cut the cost of AI training Google Cloud Run speeds up on-demand AI inference with Nvidia's L4 GPUs Nvidia to present AI and data center performance innovations at the Hot Chips conference Hotshot debuts new AI model for generating video clips Recogni's new Pareto system optimizes AI compute with minimal accuracy loss RingCentral debuts new AI capabilities for its RingCX contact center solution Dropbox acquires AI-powered calendar app Reclaim.ai There's more AI and big data news on SiliconANGLE AMD to acquire hyperscale solutions provider ZT Systems in data center AI expansion bid IT infrastructure monitoring startup Grafana Labs raises $270M at $6B valuation Eppo raises $28M in funding for its A/B testing platform Cryptography chip startup Fabric secures $33M in funding Depot raises $4.1M to expand build acceleration platform with new capabilities Snowflake beats expectations but stock falls on fears of decelerating revenue growth Palo Alto Networks shares rise following Q4 earnings beat and strong 2025 outlook Zoom impresses with second-quarter earnings beat and upbeat guidance Chip design software firm Synopsys delivers record revenue as AI accelerates demand Zuora exceeds second-quarter projections, raises fiscal 2025 revenue forecast Workday's stock flopped, then popped on confident long-term growth forecast Environmentalists raise concerns over Virginia data centers as water consumption skyrockets Rackspace expands OpenStack offerings with new enterprise-ready managed cloud solution There's plenty more news on cloud, infrastructure and apps US intelligence agencies confirm that Iran is targeting both Trump and Harris presidential campaigns Disaster recovery in action: Kaseya responds to CrowdStrike crisis Toyota alleges stolen customer data published on hacking site came from outside supplier Mandiant uncovers critical privilege escalation vulnerability in Azure Kubernetes service McDonald's Instagram hacked to promote cryptocurrency scam featuring Grimace Services at oil giant Halliburton disrupted by suspected ransomware attack Google Cloud unveils new convergence-focused security features Fortanix expands data security platform with new file system encryption feature More cybersecurity news here Apple updates iOS and iPadOS to improve compliance with EU's DMA law UK antitrust watchdog closes Google, Apple probes to revise regulatory approach Google inks controversial deal with California's lawmakers to fund local news US judge blocks FTC's ban on noncompete clauses Fintech startup Bolt reportedly raising $450M at $14B valuation Emphasis on "reportedly," since one supposed investor apparently isn't. Story raises $80M for blockchain-based IP network to address creative ownership in the AI era A man is playing video games again after Neuralink's second successful brain implant surgery HTC opens up the metaverse with Viverse Create, a no-code virtual world-building platform Wiliot brings generative AI to real-time supply chain analytics Noam Shazeer, ex-CEO of Character.AI who joined Google this month, will be Gemini co-technical lead and work with Jeff Dean and Oriol Vinyals (per The Information)