AI in healthcare has moved from pilots into daily clinical and administrative work. Physicians use it, the FDA is authorizing it at a record pace, and the rules changed again in 2026.
ONC's HTI-5 rule removed the federal model card requirement for predictive EHR tools, and Colorado replaced its AI Act with a narrower law. Lighter federal transparency did not lighten your liability. It moved the evaluation and documentation work onto health systems and the teams that build for them.
Six federal and state regimes apply, but they cluster into three questions you answer for each use case.
HIPAA governs PHI wherever it flows, including cloud and LLM providers that only store encrypted data. OCR's proposed Security Rule update would make encryption and MFA explicit. Build to the proposed controls now, since the final rule is targeted for 2027.
FDA decides whether software that diagnoses or treats is a device. A predetermined change control plan lets manufacturers pre-authorize planned model updates. ONC's HTI-5 removed EHR source attribute rules, so get model documentation through contracts and your own validation.
fairness and state law
CMS-0057-F sets 72-hour and 7-day prior authorization decisions and a FHIR API by January 2027. Section 1557 targets discriminatory decision tools. California AB 3030 and SB 1120 and Colorado SB 26-189 add patient notices and clinician-in-the-loop duties.
Classify each use case and map it to HIPAA, FDA, ONC, CMS, Section 1557 and state AI rules before you scope it.
Choose a private LLM deployment pattern, de-identify PHI, ground answers with RAG and log every inference end to end.
Validate on local data, break results out by subgroup, run silent mode and monitor drift and override rates after launch.
Move from governance to pilot to production to scale over 6 to 9 months, scored against a 10-area readiness scorecard.
No model is HIPAA compliant by itself. Compliance depends on deployment: use a vendor that signs a BAA, such as Azure OpenAI, AWS Bedrock or Google Vertex AI, restrict data retention, encrypt data, apply access controls and log usage. Consumer chatbots without a BAA should never receive PHI.
It depends on intended use. Software that diagnoses disease, analyzes medical images or drives treatment is usually a device. Decision support that clinicians can independently review may be exempt, and scheduling or billing tools generally are not devices. Get a written regulatory assessment per use case.
A PCCP is part of a device submission that describes planned model changes, the protocol for validating them and an impact assessment. Once authorized, the manufacturer can make those changes without a new submission.
California AB 3030 requires disclaimers on unreviewed generative AI clinical messages, and SB 1120 keeps clinicians in charge of medical necessity decisions. Colorado SB 26-189 takes effect January 1, 2027 with notice and disclosure duties. Track other states every quarter.
Use HIPAA Safe Harbor, which removes 18 identifier types, or Expert Determination by a qualified expert. For clinical notes, combine NLP de-identification tools with sampled human review and measure recall on your own data.
Validate on local data and break performance out by race, ethnicity, sex, age, language and payer. Check whether any input acts as a proxy for a protected trait. After launch, track drift, override rates and subgroup error rates with thresholds that trigger rollback.
A first low-risk use case, such as ambient documentation or prior authorization support, usually takes about 6 to 9 months from governance setup to production. A regulated imaging device takes much longer because of FDA premarket review.
Product and engineering leaders at US health systems and health tech companies, working alongside their compliance, legal and clinical teams.
Drop your details and we'll send AI in Healthcare: A Compliance-First Implementation Guide straight to your inbox - no spam, unsubscribe anytime.
Talk through your healthcare AI architecture and compliance plan with our engineering leads. A working session, not a sales pitch. SECTION 7 - FAQ - 5 to 8 questions
Talk to our engineers