AI in healthcare must define what should be automated before replacing human connection

Before Automating Healthcare, Define What Should Be Automated

The recent progress in AI capabilities brings us to a consequential threshold in healthcare. The question is no longer whether AI can perform aspects of the physician's role, but whether it should -- and where that boundary lies. Confronting that reality forces a deeper reckoning with what it truly means to be a physician, challenging us to examine not only the value we bring, but also the gaps, both human and systemic, that AI is poised to expose and fill. A Patient's Pain, and the System Behind It In the days before AI, I saw a 22-year-old patient who complained of lower back pain and requested a refill of a strong anti-inflammatory. As she talked, I typed the history of present illness into her EHR and asked a series of follow-up questions. Her responses became evasive, and her tone was short, suggesting that there was something important in her story she was leaving out. I stopped typing, looked up, and asked, "How is everything else going?" For three weeks, my patient had been living out of her car. She had not showered in a week, and she was struggling to find work. We talked through options, many of which had already failed her. The case manager I worked with was only available once a week and was already so overextended that my inquiries were rarely answered. My patient's back pain was real, and the healthcare system had a solution for that. But it only scratched the surface of what she truly needed to get well. An anti-inflammatory was easy to prescribe. Stability, safety, and a way back onto her life's path were not. That encounter haunts me because I was part of a system that failed her. This is what physicians in our healthcare system often encounter. Pain that begins in the body has a logical treatment. Yet what lies at the root of many ailments -- both physical and mental -- requires human empathy, connection, and time to be uncovered and eventually addressed. What could I have done -- what should I have done -- as her primary care physician? And what role could AI play in helping provide a more comprehensive and caring plan for my patient? What AI Could Have Helped Me See First, there were the logical, systemic gaps that AI could help fill, such as identifying community-based organizations that could have supported my patient while she waits to connect with the case manager or analyzing her electronic medical history to surface risk factors and unhoused status prior to our encounter. Then there is the often-cited promise of AI in healthcare: to improve workflows, freeing up clinician time and making us more present and available to focus on irreplaceable human connection. With these resources, not only could I have met the most pressing need underlying her health issues, but my day's work could have been more rewarding, more fulfilling, more human. The promise of AI in healthcare is real. Yet while everyone else is racing to implement AI that addresses many of these barriers to patient-centered care, we have not yet taken the foundational step of defining which of these capabilities should and should not be automated -- because the cost of getting this wrong is enormous. The Case for Augmentation, Not Replacement Now, 3 years after ChatGPT thrust generative AI into general use, we toggle between hype -- AI dramatically decreasing healthcare costs, for example -- and caution, with chatbots occasionally and confidently hallucinating information or omitting key details necessary to practice medicine safely. And yet, AI is already refilling medications with no human clinician in the loop. Human physicians are imperfect. So, too, are our healthcare delivery systems. AI stands ready to serve as a resource, but only if we effectively automate away the right administrative distractions and augment clinical decision-making appropriately, freeing up health workers to deliver the uniquely human components of healthcare. There is a middle ground, one where AI fills the systemic information gaps that failed my patient while preserving my ability to remain present, to empathize, and to identify the true root cause of her pain. Allison Pugh's The Last Human Job confronts this tension, highlighting the immense value in the "connective labor" undertaken in a primary care physician's office. Drawing on empathy and relying on that undefinable connection between two humans -- that is the type of labor that makes people feel seen and heard. In medicine, that connection may be the majority of what is required to reassure, educate, or motivate a patient. It's perhaps more important than the flawless delivery of perfect evidence-based care. For my patient with back pain, an ideal encounter would have included ample time for a longer discussion about her housing options, how to stay healthy during this period of transition, and resources for her to explore. And it would have had sufficient space for a more natural human interaction. AI can augment much of this interaction, but it cannot fully replace it. We Must Decide Deliberately and Without Delay Many industries and professions are grappling with similar debates over how far AI's value extends and where humans must remain. The wave of historic job losses across corporate America in October 2025 signals a disruption that healthcare institutions are not exempt from. This issue carries real financial consequences. Indiscriminate automation that fails to distinguish which tasks demand human presence from those that don't will cost governments and employers billions in worse outcomes and even more burned-out workers. Clinicians and healthcare leaders have choices to make. If we can leverage AI to do a certain task or job, should we? The answer requires urgent action. Professional societies (eg, AAFP, AMA, ANA, etc) must convene task forces tied to real-world pilots to map which clinical tasks require irreplaceable human connection and which can be safely augmented by AI. The depth and focus of AI applications in the primary care profession depend on the desired population-level outcomes and the broader context of available public health workforce and social policies. Leaders should consider all of this in context when answering the "should we" question about AI in healthcare. What Belongs to Medicine, and What Belongs to the Machine? Primary care is being asked to do more and more, without a clear definition of what physicians should uniquely be responsible for. Which is why now is the moment to clarify where the human workforce brings irreplaceable value before we automate poorly at scale. AI can help with tasks that drain our time, like filling out disability paperwork, completing EHR documentation, clicking safety screening checkboxes, and identifying supportive resources tailored to each patient and each practice. But humans must remain for the connective labor that defines our profession. A primary care physician commonly treats a patient for decades, remembering their children's sports accomplishments, their passion for baking, and, eventually, their spouse's passing. This shared history builds a deep trust that often makes the difference between a patient ignoring a challenging treatment plan and finding the motivation to see it through. We are responsible for delivering life-altering diagnoses and guiding families through difficult end-of-life discussions. And we can read the unsaid, like recognizing when a patient's tone shifts or the intuition that comes with years of experience and familiarity with a patient or their family. Feeling heard requires a trusting connection that can only come through an empathy that is transmitted from one human to another. That multidimensional feeling is impossible to sense from a flat screen. If AI handles the former, the focus can rightfully be placed back onto the human-to-human connection required for the latter. For my patient sleeping in her car, the answer to "Should we?" isn't abstract. Without making deliberate decisions about the role of AI and how it can augment the physician's role in primary care, there is risk on both sides. By not leveraging AI at the right moments and decision points, what resources and efficiencies are we leaving on the table that could drive better patient outcomes? But what do we lose by over-automating the very elements of care that matter most -- the moments of connection that patients seek and that drew us to healing professions in the first place? Technology will advance regardless. The question is whether we will be intentional enough to harness it in the service of healthier communities rather than at their expense. Travis Bias, DO, MPH, FAAFP, is a family medicine physician and deputy chief medical officer, health information systems, for Solventum. He is co-director of a comparative health systems course at the University of California, San Francisco Institute of Global Health Sciences. He has 15 years of experience across multiple clinical settings and currently practices telemedicine.

The real test for AI in healthcare is making care more human

The highest-value healthcare AI automates the administrative, not the relational, thus freeing care teams to return their attention to patients. In 1991, Mark Weiser, the then head of Xerox PARC said that the most profound technologies are those that disappear. "They weave themselves into the fabric of everyday life until they are indistinguishable from it," Weiser wrote. Today, that assertion feels more relevant - and more aspirational - than ever, particularly in the global healthcare market. The healthcare AI market is on track to reach $491 billion by 2032, growing at a compound annual rate of 43%. New tools are launching faster than health systems can evaluate them, and the promise is genuine. But when technology is deployed without a guiding purpose, efficiency quietly overtakes empathy. Too often, the rush is less about delivering better care than about competitive pressure, fear of being left behind, or the allure of a near-term return. But the leaders who will shape a responsible future for healthcare AI won't be the ones who rush to bring technology to market. They'll be the ones who pause to ask whether AI is truly delivering value - and not just the appearance of it. The ones who ask: Does this make the healthcare experience more human? The pattern is familiar: a new technology arrives, the instinct is to optimize for what's measurable while harder to quantify metrics fall by the wayside. In healthcare, that often includes relationships, continuity of care and trust. Patients experience this in real-time: chat-only front doors replacing relationship-based intake, patients at their most vulnerable routed through impersonal workflows and forced to retell their stories. Trust - healthcare's most precious currency - erodes at every step. Healthcare has been here before. The Electronic Health Record wave arrived with extraordinary promise but made things worse on the dimensions that mattered most, with studies showing that physicians now spend nearly twice as much time on EHR tasks as they spend on patient care. The lesson isn't that the technology was inherently bad. It's that deploying powerful tools without a grounding human purpose generates unintended consequences - and that in healthcare, there are no quick fixes. So how in this AI age do we avoid repeating the mistakes of the past: deploying powerful technology in pursuit of the wrong goals, over-rotating on the promise of a magical fix and wondering later why the outcomes don't match the promise? Across the industry some clear principles are emerging. The highest-value AI applications in healthcare target the rote, transactional work that was never directly serving patients or providers: authorizations, lab orders, scheduling logistics. Tasks that require speed and accuracy, not the human touch. Every administrative burden lifted from a care team is attention and presence returned to the patient. AI that filters the noise of healthcare operations creates more space for listening, empathy and human connection. What does this look like in practice? At Foodsmart, a foodcare platform, AI clears everything that gets in the way of the member relationship - the prep, the administrative load - so dietitians can focus their energy on empathy and expertise, not paperwork. The result isn't fewer humans in the loop; it's dietitians with more time to hear a patient's full story, understand their barriers to behavior change and deliver guidance that fits their life. Foodsmart's goal isn't to automate the dietitian out of the equation, but to give them their full attention back. Healthcare already generates nearly a third of all the world's data, and that volume is growing at 63% annually. Making that data useful in the moment by surfacing the right information, for the right person, at the right time is the crucial challenge. AI is uniquely positioned to solve this. When it works well, AI surfaces a patient's full context before a conversation begins - ensuring that the care team member starts informed. This matters especially in health journeys that are longitudinal rather than episodic, where each interaction is one chapter in a longer story. Fertility care is a clear example. These patients are not moving through isolated transactions - they are navigating complex, multi-step journeys that cross life stages. AI-generated summaries of prior interactions give care advocates a deeper understanding of what happened in previous conversations, so each successive touchpoint builds on the last rather than resetting. The next call is smarter. The care gets more personal over time. That continuity changes what's possible in a single moment. When a member indicates she has experienced a failed cycle, her advocate already understands her journey - the prior treatments, the emotional weight she's been carrying, what she needs right now. She doesn't have to retell her story. The conversation can begin where it should: with empathy, not discovery. When the purpose is clear and the technology is genuinely in service of it, something else becomes possible - something AI is uniquely suited to deliver, with the right human hand at the till: a system that learns how to do better over time. Every interaction generates a signal: what a patient needed, where the care fell short or exceeded expectations. AI that can capture and iterate on those signals over time and at scale doesn't just improve efficiency; it builds a progressively deeper understanding of individual patients and aggregate populations. Unlike quality assurance of the past, this isn't about measuring quality after the fact and selectively - it's about designing forward-looking systems that learn from every interaction to strengthen human connection and improve outcomes over time. Healthcare leaders from every sector are being asked to place significant bets on AI right now. But the most important measure of success won't be deployment speed or automation rates. It will be whether patients feel more understood, more supported and more connected to their care. By making the technology disappear into the experience, just as Mark Weiser envisioned, the healthcare sector can make room for the human relationships that are the foundation of good health outcomes.