Why clinical AI accuracy degrades when code sets update, how ontology mapping breaks across EHR vendors, and the canonical data layer that keeps production accuracy stable.
The Team Spent Two Months Debugging The Wrong Thing.
ICD-10-CM (68,000+ codes), SNOMED CT (350,000+ concepts), LOINC (95,000+ observations), and CPT all update on different annual or semi-annual cadences.
Ontology mapping is manual and clinic-specific — the same lab result appears as a LOINC code, free text, or a local numeric code across systems.
When code sets update, models trained on prior versions encounter codes they have never seen. Accuracy degrades silently. Customers report wrong results before engineers identify the cause.
Epic, Cerner, and athenahealth each have local mappings. The same SNOMED concept can map to different ICD codes across customers. A model that learned one mapping breaks at the next.
Notes, op reports, and radiology impressions carry the clinical reasoning. Coded data carries the billing reasoning. Models that read only the coded data are blind to the actual story.
Track which ICD/SNOMED/LOINC/CPT release each training and inference sample uses. Update impact is calculable before production rather than detectable after customer complaints.
A consistent internal representation that translates source codes from any EHR or lab vendor. Maintain version-specific mappings and flag unmapped codes for review automatically.
Structured extraction from notes, op reports, and radiology — clinical concept recognition, negation handling, temporal anchoring. Exposes the full record to the model rather than just the coded 30%.
Version tracking turns code-set updates from production incidents into scheduled pipeline reviews.
ICD-10-CM annually in October. CPT annually in January. SNOMED CT and LOINC semi-annually. RxNorm monthly. Every update is a potential production incident for any model that does not track versions.
Semi-annual hierarchy changes can return different descendant sets for the same query between releases. Cached traversals become wrong; they must be invalidated and rebuilt on each update.
A canonical representation layer that translates source codes into a consistent internal representation, maintains version-specific mappings, and flags unmapped codes for review before they cause silent accuracy degradation.
