Keeping PHI Out of LLM Prompts Under PHIPA and Bill C-27
A nurse practitioner is behind on charting. She opens ChatGPT and pastes in a rambling clinical note — patient name, date of birth, a suspected diagnosis, a medication list — and asks it to draft a clean referral letter. The letter comes back in seconds, and it's good. What she doesn't see is that the identifiable health information she just pasted is now subject to a third party's terms of use, potentially retained, processed elsewhere, or used to train the provider's models. There is no undo button. Once PHI is submitted to a public AI tool, it has left the organization's control, and PHIPA doesn't recognize "I didn't mean to" as a defence.
This is the quiet reality across healthtech and life sciences: the people closest to the most sensitive data live in browser-based SaaS all day — EMRs, trial platforms, scheduling tools — and the AI prompt box is one tab away. It is faster than a template, more forgiving than a form, and completely invisible to most of the controls a privacy officer thinks they have in place.
Why PHI is the hardest category to protect
Health information carries a combination that almost no other data type does: extreme sensitivity, dense regulation, and rapid AI adoption by the exact people who handle it. Under Ontario's PHIPA, a health information custodian is responsible for the data whether or not disclosure was authorized — and Quebec's Law 25 (with penalties up to C$25M or 4% of worldwide turnover) and the proposed Bill C-27 / CPPA framework push the same principle nationally: you are accountable for personal information the moment it leaves your control.
The accountability doesn't pause because an employee used a personal ChatGPT account on a work problem. And that's the common case, not the edge case. LayerX's 2025 research found that 77% of AI users paste data into prompts, and 82% of that pasting happens through unmanaged personal accounts. Cyberhaven's 2025 data shows roughly 40% of AI interactions involve sensitive data. When you overlay those numbers onto a clinic or a research team, the question stops being "could PHI end up in a prompt" and becomes "how much of it already has."
Consent and de-identification don't survive the paste
Health organizations lean hard on two safeguards: patient consent for defined uses, and de-identification for secondary uses like research. Both assume a controlled pipeline. A public AI prompt is not one.
Consent for treatment does not extend to disclosing a patient's record to an AI vendor the patient has never heard of. And de-identification is fragile in free text — a note that strips the name but leaves the rare diagnosis, the referring physician, and the appointment date can often be re-identified. When a researcher pastes a "cleaned" case summary into an LLM to reword it, they may be re-introducing identifiers they thought were gone, or handing over a combination specific enough to point back to one person.
The deeper problem is that these safeguards operate at the level of policy and intent, while the leak happens at the level of a keystroke. A consent form can't reach into the prompt box. This is what the Netskope 2025 finding — an average of about 223 sensitive-data policy violations per month per organization, with regulated data making up 54% of them — actually describes: the gap between what policy permits and what people do under time pressure.
Bans push the problem into the dark
The instinct is to block the tools. It rarely works. Gartner's 2026 survey found that 88% of employees with enterprise AI access also use personal AI tools for work, and 69% of organizations suspect or have evidence of prohibited public GenAI use. Ban the sanctioned tool and a clinician switches to their phone. Now the PHI still leaves, and you've also lost any visibility into it.
Governance works better than prohibition, and the numbers make the case: IBM's 2025 Cost of a Data Breach Report found that 97% of organizations with an AI-related breach lacked proper AI access controls, 63% had no AI governance policy at all, and only 17% had technical controls to redact or block sensitive data at the point of entry. Healthcare already carries the highest average breach cost of any sector at $7.42M. The organizations that fare better aren't the ones with the strictest ban — they're the ones with a control that acts before submission, because after submission there is nothing left to control.
Act before the prompt leaves, and teach in the moment
The only place to protect PHI is before it reaches the AI tool. A control that recognizes a patient identifier, a health card number, or a diagnosis in a prompt — and redacts it before the prompt is sent, substituting realistic placeholder values so the AI can still draft the referral letter — keeps the useful work while stopping the disclosure. The clinician gets her letter; the patient's record never leaves.
Just as important is what happens at the moment of the near-miss. When a prompt is stopped, a plain-language explanation of what was flagged and why turns a single risky paste into a small piece of training the person actually remembers — far more effective than the annual policy module nobody reads. Over time, that builds a workforce that's measurably better at safe AI use, and a record that demonstrates it. This is the principle Sanitized AI is built on: catch sensitive data before it's submitted, and use each stop to teach.
This quarter, ask one concrete question about your own environment: if a member of your clinical or research staff pasted an identifiable patient record into a public AI tool this morning, would you know — and could you have stopped it before it left? If the honest answer is no, that's the gap to close before a regulator asks you to describe it. If you'd like to see what acting at the prompt looks like in practice, request a demo.