AI Governance.

Key Takeaways

• Financial services and healthcare lead in formal AI governance due to regulatory pressure.
• Tech companies have built governance programs in response to enterprise customer demands.
• Common components include AI inventory, risk classification, evaluation standards, and incident response.
• The EU AI Act has driven governance investment across companies serving EU customers.
• Mature programs treat governance as enabling delivery rather than blocking it.
• Standardized tooling for governance is still maturing but improving rapidly.

Industry Examples

Financial services firms maintain detailed AI inventories and run formal model risk management programs that predate the current AI wave. Bank regulators (OCC, FDIC, Federal Reserve in the US, FCA in UK, MAS in Singapore) have issued guidance on AI use in lending, fraud detection, and compliance. The programs typically include independent model validation, ongoing performance monitoring, fairness testing across protected populations, documentation of all material decisions, and regular reviews by model risk committees.

The financial services pattern provides useful reference for other industries adopting governance. The discipline of formal model validation, ongoing monitoring, and documented governance carries over to AI specifically. Financial services firms moved into AI governance with existing model risk management infrastructure that other industries have to build from scratch.

Healthcare organizations using AI for clinical decisions face regulation as medical devices. The FDA has issued guidance on AI/ML-based medical devices including the Predetermined Change Control Plan that allows certain model updates without new approvals. Healthcare AI governance includes clinical validation, model cards documenting intended use and limitations, post-deployment monitoring, and human clinician oversight of AI-assisted decisions. The regulatory burden is significant but the patterns are well-established for organizations willing to invest.

Tech companies have built governance programs in response to enterprise customer demands. Customer security questionnaires increasingly ask about AI governance practices. Procurement processes include AI-specific clauses in MSAs. Audit firms ask for AI governance evidence as part of SOC 2 and ISO 27001 audits. The pressure has driven significant investment in AI governance at B2B technology vendors.

Insurance companies face emerging regulatory requirements (NAIC AI bulletin in the US, similar frameworks in other jurisdictions) plus traditional actuarial standards that apply to AI-driven decisions. Programs typically include fairness testing for underwriting algorithms, ongoing monitoring of pricing and claims AI, and documentation of how AI affects rate-setting decisions.

Public companies report AI use in their securities filings increasingly. The AI risk disclosures in 10-K and similar filings have become detailed enough to inform investment analysis. The disclosure requirements drive internal documentation and risk management practices.

Common Components in Working Programs

Inventory of all AI systems with classification by risk. Mature programs maintain a current list of every AI system in use including third-party AI services. Each system has metadata: owner, use case, data sources, decision impact, regulatory classification, last review date. The inventory is the foundation; without it, governance is theoretical.

Risk classification taxonomy. Systems are classified by potential for harm: data sensitivity (regulated, sensitive, internal, public), decision impact (advisory, recommendation, autonomous with reversal possible, autonomous with permanent consequences), affected populations (employees, customers, third parties, vulnerable groups), error cost (financial, reputational, physical, regulatory). Different risk levels trigger different control requirements.

Policy framework defining controls per risk level. High-risk systems get more rigorous review, more comprehensive testing, more demanding documentation. Low-risk systems get baseline controls and faster approval paths. The tiering allows the program to focus attention where it matters without creating bureaucratic gates for low-risk uses.

Model evaluation standards. Before any AI system reaches production, defined evaluation must pass. Accuracy and quality metrics. Fairness testing across user populations. Robustness testing under adversarial conditions. Security testing for prompt injection and other AI-specific attacks. The depth scales with risk level.

Production monitoring with defined metrics and alerts. Every deployed system has dashboards covering quality, drift, fairness regressions, cost, errors. Thresholds trigger alerts. On-call responders investigate. The monitoring is not optional; without it, the system that passed pre-launch review may have drifted into non-compliance silently.

Incident response procedures. AI-specific runbooks for hallucination escalations, jailbreak attempts, drift events, fairness regressions. Defined investigation steps, communication templates, regulatory notification triggers, post-incident review processes.

Documentation framework. Model cards for each system. Data sheets for each dataset. Decision logs for review board choices. The documentation is the audit trail when regulators or customers ask for evidence.

Cross-functional governance committee or review board. For high-risk systems, an independent group reviews design, evaluation, and deployment plans. The reviews are not rubber stamps; they ask hard questions and have authority to require changes or block deployment.

What Distinguishes Working Programs

Operational evidence rather than just documents. The systems are actually inventoried. The controls actually run. The monitoring actually catches issues. The incidents actually trigger documented response. When auditors or regulators ask for evidence, the team can produce it because it exists in the operational systems, not just in policy documents.

Cross-functional ownership rather than functional silos. Engineering, legal, security, product, and operations all participate. The central governance function coordinates rather than dictating. The pattern produces better outcomes than approaches where governance is owned exclusively by legal or by a separate team disconnected from delivery.

Tiered intensity by actual risk. Low-risk uses get lightweight self-certification. Medium-risk gets defined review. High-risk gets full board attention. The tiering preserves bandwidth for the decisions that actually matter rather than treating all AI uses identically.

Embedded in engineering workflows rather than separate processes. Controls run in CI/CD pipelines. Documentation generation is automated where possible. Approval workflows integrate with deployment systems. The integration produces consistent application; manual processes get bypassed.

Continuous rather than periodic. The program operates continuously rather than in periodic audit cycles. New systems get reviewed when they launch, not at the next quarterly review. Issues get addressed when they appear, not in the next compliance review. The continuous model catches issues earlier.

Common Failure Modes

Documents without operationalization. The team writes elaborate policies that describe what should happen. The policies sit in a SharePoint folder. Nothing actually changes in how systems get built, deployed, or operated. The auditor or regulator who looks at operational reality finds the gap.

Overcentralization that creates bottlenecks. The governance team tries to review every AI use. The team becomes a bottleneck. Engineering teams route around it, producing shadow AI use that is harder to govern than approved use would be. The pattern that works is tiered review with most uses self-certifying against templates.

Banning AI use rather than enabling it safely. The governance program denies most AI requests. Engineering teams use AI anyway through unapproved channels. The result is shadow AI that the governance team cannot see. Better to provide approved tools and patterns; people use AI whether allowed or not.

Skipped post-launch monitoring. Pre-launch review gets the most attention. Post-launch monitoring lags. Systems that passed review six months ago may have drifted into non-compliance silently. The discipline of ongoing monitoring is what catches drift before it produces incidents.

Insufficient documentation. When a regulator or customer asks for evidence, the team scrambles to reconstruct documentation. The reconstruction is usually poor. Documenting as work happens produces better evidence than documenting under audit pressure.

Tooling and Infrastructure

Documentation platforms for model cards, data sheets, and decision logs. Various commercial options (Weights & Biases for ML model documentation, Atlan for data documentation) plus internal tools many companies build. The format matters less than the discipline of maintaining current documentation.

Evaluation tools (Promptfoo, DeepEval, Ragas, Braintrust) for systematic AI evaluation including fairness testing. The tools provide infrastructure for evaluation work; the team still needs to define what good evaluation means for their specific systems.

Monitoring platforms (Arize, Fiddler, WhyLabs, Evidently) for production AI monitoring including drift detection, fairness regression detection, and quality tracking. The platforms reduce the engineering burden of building monitoring from scratch.

Governance-specific tools are emerging. Credo AI, Holistic AI, and similar platforms target AI governance specifically with workflows for inventory, risk assessment, and compliance documentation. The category is younger and less mature than adjacent categories but improving.

Cloud provider offerings include some governance capabilities. AWS Bedrock includes Guardrails for content moderation. Azure AI provides similar capabilities. Google Vertex AI has model cards and evaluation features. The provider-built tools handle some governance concerns within their platforms.

The tooling landscape is fragmented. Most production governance programs combine multiple tools plus internal practices. Pure off-the-shelf governance platforms exist but are less mature than the rest of the AI ecosystem.

Best Practices

• Start with inventory of all AI systems before writing policy; you cannot govern what you cannot see.
• Tier review intensity by actual risk; lightweight self-certification for low-risk uses preserves bandwidth.
• Embed governance into engineering workflows rather than running it as a separate process.
• Monitor systems after launch with the same rigor as before launch; data and models drift.
• Document as you go rather than under audit pressure.

Common Misconceptions

• AI governance is the same as compliance; compliance is one outcome of good governance, but governance is the broader system that produces compliant results by design.
• A signed policy document is enough; what matters is whether controls actually run when systems launch.
• Governance slows AI delivery; well-designed governance speeds delivery by making approval predictable.
• Only highly regulated industries need formal AI governance; enterprise customer demands and reputational risk now require it more broadly.
• Banning unapproved AI use eliminates the risk; in practice it creates shadow AI that is harder to monitor than approved use.