Automation is a pervasive part of today’s healthcare landscape. Automation uses emerging technologies, including AI, robotics, and data analytics to communicate with patients, optimize billing, improve diagnostics, provide more individualized treatment, reduce manual labor, and streamline workflows.

The U.S. Department of Health and Human Services is developing a strategic plan to implement President Biden’s 2023 Executive Order on the “Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence.” As mandated by that Order, HHS is to advance “responsible AI innovation” in the health and human-services sectors by establishing policies and guidance (and proposing regulatory action, as appropriate) related to the use of AI in scientific research and discovery, drug and device development, disease prediction, healthcare delivery, and public health. 

The Executive mandate’s directive to HHS focuses on ensuring that new healthcare products, tools, and infrastructures are safe, unbiased, and protect the well-being of individual patients, caregivers, and other stakeholders, as well as the population at large. In undertaking this critically important work, the U.S. joins the EU and others in developing ethical guidelines for the use of AI systems. Yet, neither Biden’s order, nor Europe’s AI Act, address some of the central issues surrounding the use of AI in healthcare or the automation of healthcare more generally. These issues include how care—and the labor of care–is valued, assessed, distributed, and understood.

The public is skeptical about the contributions AI will make to healthcare. A 2023 survey revealed that fewer than half of U.S. adults expected AI to improve health outcomes. More than half worried that AI would worsen the patient-provider relationship. Yet, the healthcare automation market is booming, as tech companies promise that AI and data platforms will reduce cost inefficiencies and administrative burdens. As this suggests, automated healthcare contributes to the (ongoing) commodification of care, giving both care and caregivers a dollar value. This dollar value comes to dominate the social value of care, as both the form and content of care are adapted to maximize profits. 

Related to profitability motives, the automation of care shifts the assessment of care from markers of quality to markers of efficiency. To ensure such efficiencies, the use of AI in healthcare systems care mimics a Fordist model of production where assembly-line manufacturing systems break down product processes into small, repetitive, standardized, tasks. The reduction of care to assembly line processes shifts both the ways in which care labor is distributed and the meaning of care itself.

Some companies promise that generative AI will improve healthcare workflow through automating tasks such as diagnostics (from hip fragility to cancer diagnoses), care management (from emergency medical care to the compounding of pharmaceuticals and dispensing of medication), record-keeping and analysis (from data health reasoning to financial records) and customer interface (from patient portals and chatbots to customer experience surveys and grievance redress systems).

 While health care executives laud these tools, physicians worry about lax oversight and nurses point toward their increased administrative burdens, as new workflow systems have them “nursing the computer rather than nursing the patient.” In addition to contributing to nursing burnout, automated workflow systems may displace the labor of care from healthcare workers to patients themselves. Digital care pathways may help patients and their families coordinate care with health care professionals. At the same time, the introduction of digital tools may result in outsized care burdens for patients and their families. Automated workflows ask a patient—who may lack both the medical and the technological knowledge–to find an appropriate specialist, create an online account, install healthcare apps on a smart device, upload and update medical history, scan insurance cards, sign medical consent documents, obtain and engage wearable medical devices, monitor and report symptoms, complete patient experience surveys, and more. Where patients are very young, very old, sick, or disabled, such “person-centered care” shifts a significant burden of care onto a patient’s family.

It remains to be seen whether planning around so-called  “ethical” AI— in the U.S. or in other parts of the world—will result in improved systems of healthcare. It is possible that improvements can be made to oversight, algorithmic biases, and infringements of privacy. But such improvements, however desirable, overlook a fundamental conundrum involved in the very idea of “automated care.”

 Consider what we mean when we say that a human is on “auto-pilot.” The trucker who cannot remember how she got to her destination may be criticized for being less than careful. Similarly, the doctor, nurse, mother, friend, or lover who is simply going through the motions (however fine-tuned and accurate those motions may be) seems an uncaring participant in the relationship she undertakes. What, then, do we mean when we talk about automated care?

To say that something is “automatic,” conveys that it may be involuntary (unwilled), impulsive (irrational), spontaneous (instinctive), knee-jerk (activated without deliberation), mechanical or robotic (lifeless). Automated billing is one thing. But the idea of care as an automated product or service drains care it of its value and obscures the affective, embodied, relational, skilled, deliberative, and creative components of care work (both inside and outside of health care settings). By so doing, it fails to heed to the lessons of several decades of feminist work aimed at helping us understand the nature and moral significance of care. 

The cover picture is designed by Nancy Folbre
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.