An AI model rejects a loan application. The applicant, who would have been approved under the previous manual process, is denied credit. The denial is based on a pattern the model identified in the applicant’s transaction history that correlates with default risk in the training data.
The applicant asks: why was I rejected?
The loan officer does not know. The model produced a risk score. The score was below the threshold. The officer followed the process. The model made the decision.
Who is responsible? The loan officer who applied the score? The data scientist who built the model? The risk committee that approved the threshold? The vendor who provided the algorithm? The board that authorised the deployment?
The answer, in most organisations, is unclear. And the lack of clarity is not a governance oversight. It is a governance gap: the organisation deployed a decision-making system without defining who owns the decision, who owns the model, who monitors it and who can override it.
Where the Gap Appears
Pricing decisions
An AI pricing engine recommends price adjustments based on demand signals, competitor pricing and customer behaviour. The engine adjusts prices for 50,000 SKUs daily. No human reviews the individual adjustments. The engine misprices a product category for three weeks, generating Rs 80 lakh of margin erosion before the error is detected.
Who approved the engine’s authority to adjust prices without human review? Who monitors the engine’s output for anomalies? Who is accountable for the margin loss?
Fraud detection
An AI fraud-detection system flags 200 transactions per day. The operations team reviews the flags. The team clears 190 as false positives. Ten are escalated. But the team, overwhelmed by volume, reduces its review rigour over time. A genuine fraud passes through the system unflagged because the model’s parameters were calibrated 18 months ago and have not been updated.
Who is responsible for keeping the model current? Who measures the false-positive rate and adjusts the threshold? Who ensures the review team maintains investigation quality?
Employee decisions
An AI-assisted hiring tool screens 500 applications and ranks the top 50. The hiring manager interviews from the ranked list. The tool’s ranking methodology, trained on historical hiring decisions, may embed biases from past hiring patterns. The organisation hires from the AI’s recommended list without understanding why those candidates were ranked higher.
Who validated that the model does not discriminate? Who audits the ranking methodology? Who is accountable if the tool’s recommendations produce a workforce that lacks diversity?
The AI Decision Accountability Matrix
In Northrop Management Private Limited’s governance advisory practice, AI governance is structured through a decision accountability matrix that defines ownership at every layer.
Layer 1: Model ownership. Every AI model has a named owner: the individual or function responsible for the model’s design, training data, methodology and performance. The owner is accountable for the model’s output quality.
Layer 2: Deployment authority. Before an AI model is deployed in a decision-making process, a defined authority (the risk committee, the CTO, the relevant business head) approves the deployment, the scope of decisions the model can make autonomously and the conditions under which human override is required.
Layer 3: Human-in-the-loop requirements. For each AI-assisted decision category, the matrix defines: can the model decide autonomously (no human review), or must a human approve the model’s recommendation before it takes effect? The answer depends on the consequence of error: a mispriced SKU is correctable. A denied loan application has immediate customer impact. A fraud flag that triggers an account freeze has legal implications.
Layer 4: Monitoring and validation. Every deployed model is monitored for: accuracy (is the model’s output still correct?), drift (has the model’s performance changed over time?), bias (does the model produce systematically different outcomes for different groups?), and relevance (has the business context changed in ways the model does not capture?).
Layer 5: Override and escalation. When a human disagrees with the model’s recommendation, a defined escalation process determines: who can override, what documentation is required and how overrides are tracked to identify patterns that suggest the model needs recalibration.
Layer 6: Audit trail. Every AI-assisted decision is logged: the input data, the model version, the output, the human action (approve, override, escalate) and the outcome. The trail enables retrospective review when a decision is challenged.
The Governance Standard
AI governance is not: “Do we have an AI policy?”
It is: “For every AI-assisted decision in this organisation, can we identify: who owns the model, who approved its deployment, who monitors its performance, who can override it, who is accountable for its errors and whether the decision trail is auditable?”
If the answer to any of these questions is “nobody” or “unclear,” the organisation has a governance gap that increases in consequence as the AI’s decision authority increases.
Ashish Chaudhary, frames the governance requirement directly: “AI does not make decisions. It produces recommendations that the organisation acts on. The governance question is not whether the AI is accurate. It is whether the organisation knows who is responsible when the AI is wrong, who monitors whether the AI is drifting, who can override the AI when judgment demands it and whether every AI-assisted decision can be reconstructed and explained. An algorithm without accountability is not intelligence. It is uncontrolled automation.”
Questions for the Boardroom
- For every AI model deployed in a decision-making process, can we name the individual who owns the model and is accountable for its output?
- Which AI-assisted decisions in our organisation can be made autonomously (without human review), and who approved that authority?
- How frequently are our deployed AI models validated for accuracy, drift, bias and relevance?
- If a customer, a regulator or a court challenged an AI-assisted decision, could we reconstruct the decision trail (input data, model version, output, human action, outcome)?
- Has the board been briefed on which decisions in the organisation are AI-assisted, what governance framework applies to each and what the consequences of model failure would be?
Closing Implication
AI governance is decision governance applied to algorithmic systems. The governance gap exists when the organisation deploys AI in its decision processes without defining the accountability, monitoring, override and audit mechanisms that any consequential decision-making process requires. The gap is invisible when the AI is accurate. It becomes visible, expensively, when the AI is wrong and nobody can explain who was responsible, what went wrong or how to prevent it from happening again.