Shadow AI in a hospital or health system rarely looks like the classic image of an employee secretly running a side project. It looks like a clinician pasting a patient’s history into a consumer chatbot to draft a note faster, a department subscribing to an AI scribe tool without looping in privacy or IT, or an EHR vendor shipping a new AI feature that turns itself on for every tenant on the next update. None of it requires malice. All of it can put protected health information (PHI) somewhere the organization never approved and cannot account for.
Working through this in production? See how we run a healthcare AI governance path.
This is the healthcare-specific version of shadow AI discovery: the same underlying problem covered in DSE’s general shadow AI inventory guide, but with the sources, consequences, and inventory priorities that are specific to a healthcare environment.
The three places healthcare shadow AI actually comes from
1. Clinical and administrative staff using consumer AI tools
The most common source is also the most mundane: a clinician, care manager, or administrative staff member uses a general-purpose AI assistant to draft a note, summarize a case, or answer a documentation question, and pastes in enough patient context to make the answer useful. No procurement happened. No BAA exists. The tool was never evaluated for PHI handling because nobody framed it as a PHI-handling decision at the time.
2. AI features that arrive inside tools you already have
Healthcare organizations run on a small number of large platforms, principally the EHR, and those platforms increasingly ship AI features as part of routine updates. An ambient documentation assistant, a suggested-coding feature, or a patient-messaging draft tool can activate for an organization without a distinct purchase decision, which means it can also bypass the review that a new vendor relationship would normally trigger. The organization technically already has PHI flowing through the platform. The question shadow AI discovery has to answer is whether the new AI layer inside that platform changes where the data goes, who can see it, and what subprocessors are now involved.
3. Department-level pilots that outrun governance
A single department, often one under pressure to reduce administrative burden, adopts an AI tool for its own workflow: prior authorization drafting, care-management triage notes, patient-message drafting. The pilot works well enough that it becomes standard practice long before privacy, compliance, or security ever reviews it. By the time it surfaces in a formal inventory, it may already be load-bearing for the department, which makes it harder to unwind and easier to rationalize as low-risk.
Why the standard is different once PHI is involved
In most industries, an unsanctioned AI tool is primarily a data-governance and IP-leakage problem. In healthcare, the moment PHI is involved, an ungoverned AI use case is a HIPAA gap by definition, independent of whether anything has gone wrong yet. The absence of a business associate agreement, the absence of a minimum-necessary scope, and the absence of audit controls are themselves the exposure. A tool can go a year without an incident and still represent live regulatory and breach-notification risk the whole time, because the standard is about the presence of appropriate safeguards, not the absence of an observed failure.
That reframing changes how urgently a healthcare organization should treat shadow AI discovery relative to other sectors. It is not a hygiene exercise. It is closing a standing compliance gap.
What a healthcare-specific shadow AI inventory should capture
A general AI inventory asks what tool, what department, and what data. A healthcare inventory needs three additional fields to be useful for a HIPAA readiness program:
| Field | Why it matters in healthcare |
|---|---|
| PHI exposure type | Whether the tool touches identifiable clinical data, de-identified data, or no patient data, since this determines whether a BAA is even required |
| BAA status | Whether a business associate agreement exists and whether its scope actually covers the AI feature, not just the base product |
| Clinical workflow proximity | Whether the tool sits inside a documentation, coding, or patient-communication workflow versus a purely administrative one, since clinical proximity raises both risk and review priority |
Recording these three fields turns a generic tool list into something a privacy officer or compliance lead can act on immediately: which findings need a BAA conversation this week, and which are lower-priority administrative tools that can wait for the next review cycle.
Finding it: the channels that matter most in a healthcare environment
The full multi-channel discovery methodology (network and SaaS-usage inventories, OAuth-grant review, expense records, and voluntary employee surveys) is covered in the general shadow AI discovery guide and applies here too. Two channels carry more weight in a healthcare environment specifically:
- EHR vendor release notes and admin console review. Since the EHR is the platform most likely to introduce AI features without a distinct procurement event, someone needs to own reviewing vendor release notes and the EHR admin console for newly enabled AI capabilities, not just new standalone tool purchases.
- Department-level, non-punitive interviews. Because department pilots are often adopted informally under workload pressure, a survey alone tends to undercount them. A short, explicitly non-punitive conversation with department leads, framed as inventory-building rather than enforcement, surfaces tools that a system-wide survey misses.
Turning the inventory into action
Once a healthcare shadow AI inventory exists, the next steps follow the same governance program covered in HIPAA AI governance readiness: confirm or establish the BAA, define minimum-necessary scope, map Security Rule safeguards, and decide whether the use case can continue on its current path or needs to move to a governed replacement. The inventory is the starting artifact. The readiness program is what makes the finding defensible.
What this guide is / What it is not
What it is: a healthcare-specific guide to where shadow AI comes from and what to capture when inventorying it, so a HIPAA governance program has real findings to act on. What it is not: legal advice, a certification, or a guarantee of any HIPAA audit or breach-notification outcome. DSE helps healthcare organizations build and act on this inventory. We do not certify HIPAA compliance and do not guarantee any regulatory or examination result.
FAQ
Is a clinician using a consumer AI chatbot to draft notes considered shadow AI?
Yes, if the organization has not evaluated, approved, or established a business associate agreement for that tool and patient information is included in the prompts. It does not require malicious intent to count as shadow AI. The absence of governance and a BAA is what defines it, not the user’s motive.
Do AI features that an EHR vendor adds automatically count as approved, since we already have a BAA with that vendor?
Not automatically. A new AI feature can route data to a different subprocessor or model provider than the base EHR agreement contemplated. It needs to be confirmed as covered, not assumed, which is why EHR vendor release notes and admin console changes are a priority discovery channel in healthcare specifically.
Is an ungoverned AI tool touching PHI automatically a HIPAA breach?
Not automatically a reportable breach, but it is a governance gap the moment it is discovered. The missing business associate agreement, undefined scope, and absent audit controls are themselves the exposure, independent of whether any specific disclosure has occurred. Closing the gap is the priority regardless of whether an incident has happened yet.
Why does a healthcare AI inventory need different fields than a general shadow AI inventory?
Because the action a healthcare organization needs to take depends on three healthcare-specific factors: whether the tool touches identifiable PHI, whether a business associate agreement exists and actually covers the AI feature, and how close the tool sits to a clinical workflow. Recording those fields lets a privacy officer prioritize findings instead of just cataloging tools.
What is the most commonly missed source of healthcare shadow AI?
AI features that activate inside an existing platform, most often the EHR, through a routine vendor update rather than a new purchase. Because no new procurement event occurs, the standard review process that would normally flag a new vendor relationship is often bypassed entirely.
DSE’s Shadow AI Discovery + Policy Readiness Sprint builds this inventory for healthcare organizations and turns it into an exposure map, a prioritized control roadmap, and an AI acceptable-use policy framework. It pairs with HIPAA AI governance readiness for the program that follows the inventory. Scope the engagement →
Key facts
- Healthcare shadow AI has three distinct sources that general shadow-AI discovery often misses: consumer AI tools used for clinical drafting, AI features that activate inside the EHR through a vendor update with no local procurement event, and department-level pilots that never received privacy or compliance sign-off (DSE, 2026).
- A shadow AI use case involving PHI is a HIPAA governance gap the moment it is discovered, independent of whether any data has actually been disclosed improperly, because the missing business associate agreement and access controls are themselves the exposure (DSE, 2026).