AI Employees Undermine Work Quality as Companies Rush to Deploy Digital Coworkers

AI agents are not your "coworkers"

This story originally appeared in The Algorithm, our weekly newsletter on AI. To get stories like this in your inbox first, sign up here. Imagine coming in to work to learn that a new underling will report to you. The worker is not a person but an AI tool -- one that your company nonetheless calls Alex, an "employee" with a title and defined responsibilities. How well do you think you would work with Alex? If you're anything like the managers recently studied by Emma Wiles, a Boston University business professor, treating Alex as a "coworker" and not a software tool would lead you to do a worse job. Wiles found that people caught 18% fewer errors when the work was said to have come from an agentic "AI employee" rather than a chatbot. It turns out that what's in a name matters. A lot. This is an alarming glimpse of the future Silicon Valley is hurling us toward. Last year Nvidia's CEO, Jensen Huang, talked about workplaces of "digital humans." Since April, Microsoft, OpenAI, Anthropic, and Google have all released new tools oriented toward managing teams of AI agents, many of which are explicitly advertised as digital colleagues with the flexibility and cognitive power of actual humans. And nearly a third of the 1,261 managers who participated in Wiles's study said their companies already frame AI agents as employees (23% even list them on org charts). The technical progress of agentic AI is not all hot air, of course. Agents, which can effectively be thought of as AI tools programmed to work in a loop until they achieve a goal, have become measurably better at more complicated tasks. But it's a huge leap to refer to these tools as coworkers or employees, and doing so will set unrealistic expectations for what AI can do while leaving the human employees supposedly responsible for them worse off. That's partially because, Wiles's research suggests, it inverts our sense of who's in charge. When an AI tool was framed as an employee, participants in the study saw themselves as less responsible for its output. They were also 44% more likely to escalate its questionable work to a manager for further review rather than trusting their own corrections (thus negating the time-saving purpose of using the AI agent in the first place). That matters far beyond office culture: As AI agents are embedded into health care, warfare, education, and government, there's a growing risk they'll become a convenient place to dump blame for failures that are instead the product of bad human decisions, incentives, and oversight (recall how the bomb strike on a girls' school in Iran was popularly blamed on Claude, when all signs point to a cascade of human errors). "AI agents right now are being marketed as things that can replace humans, and I think that's just a losing proposition," says Daron Acemoglu, an economist at MIT who won the Nobel Prize in 2024 and studies AI's impact on the economy. "They should instead be optimized so that they can improve human capabilities, which is not what they have [been] at the moment." What could that look like? Consider a new effort at Stanford, where researchers presented 1,500 workers in 104 jobs with information about what tasks AI could potentially do in their work and then asked what would actually be most helpful and productive. Workers did want automation in certain areas: Law clerks thought AI could help ensure that adequate progress was being made across cases, for example. But often the tasks that tech experts deemed most suitable for AI -- like verifying customer credit ratings for sales reps -- were what the actual workers said they definitely did not want or need an agent to do. Which brings us back to Alex. Calling Alex an employee is easy -- and convenient, especially when something goes wrong -- but it's a branding exercise. It doesn't make the tool more fit for the job, and as Wiles's research shows, it makes the humans around it worse at theirs. And recall that they are the ones with the agency that AI is trying to replicate. They deserve better than Alex.

A.I. 'Employees' Might Disrupt Work in Unexpected Ways

Scholars say the "unknown unknowns" of using artificial intelligence in the workplace may be undermining the technology's advertised benefits. Over the past year or two, companies have started using so-called artificial intelligence agents as bona fide "employees," even including them in their organizational charts. Emma Wiles, a Boston University professor who studies how A.I. affects workers, stumbled onto this phenomenon in October, at a conference where two human resources executives said that treating A.I. agents like real employees was a way to increase productivity and to put their companies on the cutting edge. But when Dr. Wiles and three collaborators from Boston Consulting Group investigated further, they discovered a pitfall. In an experiment involving dozens of companies with A.I. employees, the researchers found that managers tended to vet documents less carefully when told an A.I. employee had produced them. The managers missed errors that other managers caught when told they were vetting the work of a human. Dr. Wiles speculated that managers didn't think sussing out mistakes made by A.I. employees was their responsibility. If something went wrong, they could dismiss it as the fault of the tech team, or of the executives who wanted A.I. employees in the first place. "But it's not your problem," she said, channeling the managers' mind-set about their own roles. In the years since A.I. burst onto the scene, many companies have become aware of flaws produced by the technology and, at times, taken steps to offset them. They know that A.I. models can be biased against certain groups of people, like nonwhites. They know that chatbots can provide confident but incorrect answers to queries. They know that the bots sometimes spill the beans on information that should remain private. But as companies race to bring A.I. into their day-to-day operations, researchers are discovering more subtle defects. In principle, these flaws could be corrected, too. For example, companies could hold managers directly responsible for the mistakes of A.I. subordinates. But in practice, most corporate users appear to be blissfully unaware of these issues, raising the possibility that A.I.'s promise of increased productivity and vast cost savings could be undermined. Even researchers who study A.I. may be aware of only a fraction of the problems that the technology introduces. "There are a whole host of unknown unknowns," Dr. Wiles said. One well-documented but underappreciated flaw of artificial intelligence models is that they tend to favor work produced by artificial intelligence. A 2025 paper in The Proceedings of the National Academy of Sciences found that several large language models had a low opinion of text written by humans, creating a "potentially consequential form of implicit 'anti-human' bias." But many companies seemed unaware of this problem, or at least unable to imagine how it might wreak havoc on their operations. When a team of scholars spelled it out in a subsequent paper, finding that the A.I. models that companies use to evaluate résumés tend to favor those written with the help of A.I. over those written entirely by humans, it caught the attention of some corporate recruiters. Jane Yi Jiang, an operations professor at Ohio State University who is an author of that subsequent paper, said that she and her co-authors were happy to help when recruiting firms inquired about "how to improve their processes." But they noted that this was almost certainly not the only problem companies were inadvertently introducing in their rush to adopt A.I. "People are moving so fast to use L.L.M.s without thinking too much about the implications, biases," she said. For example, some companies now use A.I. to help answer questions like how much to charge for a product, or where to open a new location. Relying on the technology for such purposes, however, can quickly go off the rails. When left to their own devices, humans often cooperate and seek win-win outcomes. But when A.I. models assess a situation, they tend to adopt the more coldly calculating, "rational" mind-set that arises from basic game theory. They might, say, lead a company to aggressively undercut a competitor, even though it risks a damaging price war. "Most of the L.L.M.s we test think that human beings are more rational than they actually are," said Jiannan Xu, a Ph.D. candidate at the University of Maryland and collaborator of Dr. Jiang's. "But the most rational response leads to a bad situation for all" in many cases. In principle, developers and users of A.I. can correct for these biases. Dr. Jiang and Mr. Xu, for instance, found they could reduce anti-human bias by simply instructing models to focus on the quality of the written material they evaluate, and to avoid considering the author. But A.I. researchers can't correct for biases they aren't aware of, and several scholars said the impact of these undetected biases could grow. One way is if future models are trained on data produced by today's models without sufficient care, creating a kind of self-reinforcing loop. In that case, "the tendency to consolidate on existing perspectives and behaviors seems likely," said Shayne Longpre, an A.I. researcher and founder of the Data Provenance Initiative, a group that monitors A.I. infrastructure. And then there are the blind spots that arise not so much from A.I. itself, but from the way humans use it. Scholars who turn to A.I. at every stage of the research process -- asking A.I. what questions are worth studying; seeking its advice on how to answer these questions; enlisting it to analyze data; relying on it to help write up findings -- could inadvertently narrow the scope of their work. "We don't necessarily notice it at the individual level," said Cecilie Steenbuch Traberg, a psychologist at the Copenhagen Business School and an author of a recent paper on the topic. "You're sparring with a chatbot, it's helping me come up with ideas, you might think it sounds great. But at the collective level, it looks pretty similar. Everyone is sounding alike." Dr. Wiles, the Boston University professor who examined the way humans manage A.I. employees, said the shortcomings weren't necessarily intrinsic to the technology, but arose when humans adopted it with little attention to what could go wrong. She and her colleagues surveyed more than 1,000 corporate managers, and found that about one-third said their organizations referred to A.I. as a "teammate or employee," and that nearly one-quarter said their employer included A.I. agents on its organizational charts. "We call it Scout," one manager told the researchers in an interview, referring to an A.I. agent. "It's technically an equivalent peer on your team." Dr. Wiles and her colleagues gave all the managers they surveyed a set of five documents that contained errors, and gave them 20 minutes to review as many as possible. In some cases they told the managers that an A.I. employee had done the work; in some cases they said that an A.I. tool had done the work; and in some cases they said that a human had done the work. In general, the stated source of the documents didn't make much of a difference in how closely managers vetted them. But managers at companies that included A.I. agents on their organizational charts caught substantially fewer mistakes when told they were reviewing the work of an A.I. employee. People who manage humans tend to assume that "if someone on my team makes a mistake, that's on me," Dr. Wiles explained, which is why they closely check the work of these subordinates. Managers also seem to assume that they're on the hook for work produced by an inanimate A.I. tool. But managers at companies with A.I. employees don't seem to feel the same responsibility for the work of those employees. Her takeaway: Over the past few centuries, scholars and business leaders have developed a reliable set of practices for managing humans. But the psychology of managing anthropomorphized A.I. is vastly different, and "we're going out there blind." She worries that the problem is about to get worse. At the same conference where she first heard H.R. officials talk up the virtues of their A.I. employees, one went even further, saying her company would soon have A.I. employees managing humans. "A hush went over the room," Dr. Wiles recalled. "We'll need someone to study that, too," she added.