Ask who is responsible for a decision an AI system made, and the answer you must never accept is "the model". A model is an artefact; it cannot be accountable, cannot be sanctioned, and cannot stand before a regulator. Accountability has to rest with people and functions. The work of governance is to make sure that, for every consequential decision your systems make, there is a specific human or role who owns it — and who has both the authority and the information to discharge that ownership. This part is about building that structure so it is real, not decorative.
The principle: no orphan decisions
The organising principle of AI governance is simple to state and hard to live by: no decision should be an orphan. Every automated decision of consequence must trace to an owner. That owner is accountable for whether the decision was appropriate, has the authority to change or stop the system, and can be expected to answer for it internally and externally.
This sounds obvious until you look at how AI systems are actually built. A data science team trains a model; a platform team deploys it; a product team wires it into a workflow; a business unit consumes its outputs. When something goes wrong, each can plausibly point at another. Governance exists to dissolve that ambiguity in advance, before the failure, by assigning ownership explicitly.
Diffuse accountability is the natural state of a complex system. Governance is the deliberate work of making it specific.
The three lines of defence
The most widely used structure for organising this — borrowed from financial risk management and well understood by regulators — is the three lines of defence model. Its value is that it separates the people who take risk from the people who oversee it from the people who independently assure it.
First line: the business that owns the risk
The first line is the function that builds, deploys, and profits from the AI system — the business owner, supported by data scientists and engineers. They own the risk because they own the system. Their job is to operate it within the controls and limits set for it, to monitor it day to day, and to escalate when something looks wrong. The accountable owner of an AI decision lives here. Critically, the first line is not a passive recipient of rules handed down from compliance; it is the primary owner of risk and the first to act when the system misbehaves.
Second line: independent oversight
The second line — risk and compliance functions — sets the framework the first line operates within and independently challenges its decisions. They define the classification scheme, the validation standards, the documentation requirements, and the risk appetite. They review and challenge classifications and validations rather than rubber-stamping them. The independence matters: the second line must be able to say no to the business, which means it cannot report to the business it oversees.
Third line: independent assurance
The third line — internal audit — provides independent assurance that the first two lines are actually doing their jobs. They do not run controls; they verify that controls exist, are designed well, and operate effectively. The third line answers the question the board ultimately cares about: can we trust that our AI governance is real, or is it theatre?
The model is not bureaucracy for its own sake. Its purpose is to ensure that the people taking risk are not the only people checking it, and that someone independent confirms the whole thing works.
Key roles around an AI system
Within that structure, certain roles recur around any significant AI system. The titles vary by organisation; the responsibilities do not.
- The accountable owner. A named senior individual — usually a business leader — who is answerable for the system's decisions. This person does not need to understand the gradient descent, but must understand what the system does, what could go wrong, and what controls protect against it. When a regulator asks "who is responsible for this?", this is the name.
- The model developer. The data scientists and engineers who build and maintain the system. They are accountable for sound technical construction, honest documentation, and surfacing limitations rather than hiding them.
- The independent validator. Someone outside the development team who critically assesses the model before it goes live and on an ongoing basis. Independence is the whole point — a validator who reports to the developer is not a validator.
- The risk and compliance reviewer. The second-line function that confirms the system meets the framework's requirements and falls within risk appetite.
- Operational staff. The people who run the system day to day, handle its outputs, and exercise any human oversight. Their role is often underestimated, yet they are the ones who first notice when something drifts.
Making ownership real
The failure mode of governance is ownership that exists on paper but not in practice — a name in a document who has neither the information nor the authority to actually own the system. Several conditions make ownership real rather than nominal.
The owner must be informed
An owner who cannot see how their system is performing cannot own it. Ownership requires a flow of information to the owner: performance against expectations, drift, override rates, incidents, complaints. A dashboard the owner actually reads beats a quarterly report nobody opens.
The owner must have authority
Accountability without authority is a trap — holding someone responsible for a system they cannot change or stop. The accountable owner must have the genuine power to pause the system, demand changes, or withdraw it. If pulling the system requires fighting three other departments, the ownership is fictional.
The owner must have capacity to understand
Owners need enough understanding to exercise judgement — not the mathematics, but the system's purpose, its limitations, its failure modes, and the meaning of its controls. Part of the governance function's job is to equip owners with that understanding, in plain language. An owner who signs off on something they do not comprehend is a control in name only.
Governance bodies and escalation
Individual roles need a forum. Most institutions stand up some form of AI or model governance committee — a cross-functional body that approves high-risk systems before launch, reviews their ongoing performance, and adjudicates difficult cases. Done well, this committee is where the obligation map, the classification, and the validation come together for a genuine decision. Done badly, it is a rubber stamp that meets monthly to approve whatever the business brings it.
The difference is whether the committee has teeth: real authority to reject systems, genuine independence in its membership, and a documented record of decisions and their reasons. Equally important is a clear escalation path — when an operator on the front line sees a system misbehaving, they must know exactly how to raise it and trust that doing so leads to action rather than blame.
Documenting the governance
As with everything in this discipline, governance that is not documented is, for practical purposes, governance that does not exist. For each significant system you should be able to produce, on demand: who owns it, who built it, who validated it, who approved it, and when each of those happened. This is not bureaucracy for its own sake; it is the record that lets the institution — and a regulator — see that accountability is assigned and real. The next several parts build on this foundation, starting with how the mature discipline of model risk management adapts its proven machinery to the particular challenges of AI.
When the three lines blur — and why it matters
The three-lines model is only as good as the independence between the lines, and in practice that independence is constantly under pressure. A few failure patterns recur often enough to name.
- Validation captured by the business. When the second line reports, even informally, to the people whose systems it validates, its challenge softens. The validator who depends on the business for budget, headcount, or goodwill finds reasons to approve. The remedy is structural: genuine separation of reporting lines, so saying no carries no personal cost.
- The committee as rubber stamp. A governance body that approves everything brought to it is not governing; it is laundering decisions made elsewhere. The tell is the absence of any record of rejection or material challenge. A committee that has never said no, or never sent something back, is almost certainly not exercising real authority.
- Audit without teeth. A third line that reports issues which are then never remediated provides assurance theatre. Effective assurance requires that findings drive change, tracked to closure, with escalation when they are ignored.
The structure exists precisely to resist these pressures, but structure alone does not guarantee independence; it has to be defended actively, and the clearest evidence that it is real is a documented history of the lines actually challenging one another.
The accountable owner in practice
Earlier we said the accountable owner must be informed, empowered, and able to understand. It is worth being concrete about what that looks like day to day, because the role is easy to assign and hard to make real.
An informed owner receives — and actually reads — a regular, digestible view of how their system is performing: its key metrics against expectations, its drift indicators, its override and complaint rates, and any incidents. Not a 60-page quarterly report nobody opens, but a living picture they engage with. An empowered owner can, without fighting a political battle, demand changes to the system, tighten its controls, or pause it outright — and everyone involved knows that authority is real. An owner able to understand has been equipped, in plain language, with what the system does, where it is weak, what could go wrong, and what each control protects against; they do not need the mathematics, but they must be able to ask intelligent questions and recognise a wrong answer.
Test your ownership honestly: if this system started making harmful decisions tomorrow, would the named owner know quickly, and could they stop it without anyone's permission? If not, the ownership is on paper only.
Scaling governance across a portfolio
Individual roles and a committee work for a handful of systems. Most institutions soon have dozens or hundreds, and governance must scale without either collapsing into bureaucracy or thinning into nothing. The reconciling principle is, once again, proportionality applied through the structure. The full apparatus — independent validation, formal committee approval, intensive oversight — is reserved for the high-risk tier. Lower tiers follow lighter, often partly automated, governance paths with standardised templates and self-certification subject to audit. The committee's agenda is curated so that its scarce attention goes to the genuinely consequential and contested cases, not to routine approvals that a defined process can handle. And the model inventory (the subject of the next part) is what makes this scaling possible at all: it is the single place where the whole portfolio is visible, so that governance can be directed by risk rather than by whoever happens to shout loudest. Governance that does not scale ends one of two ways — either it becomes a bottleneck the business learns to route around, or it quietly stops applying to most systems. Designing it to scale proportionately from the start avoids both fates.
In the next part: model risk management — how a discipline built over decades for traditional models extends to machine learning, and where AI breaks its assumptions.