Every regulated-AI regime, and every workable internal framework, rests on the same foundational move: classify the system by the harm it can cause, then govern it in proportion to that harm. Get the classification right and everything downstream — the depth of validation, the intensity of monitoring, the level of human oversight — follows naturally. Get it wrong and you will either smother low-risk systems in needless process or, far worse, wave a high-risk system through with controls built for something trivial.
This part is about doing classification well: the dimensions that determine a tier, a workable tiering scheme, the traps that lead to mis-classification, and how to document a classification so it survives scrutiny.
Why classification is the load-bearing decision
Classification is load-bearing because it is the single input that scales all your other effort. A risk function cannot apply maximum rigour to every model — there are not enough hours, and doing so would grind the business to a halt. Nor can it apply minimum rigour to all of them without inviting disaster. The only sustainable answer is differentiation: spend your scarce, expensive governance capacity where the stakes justify it. Classification is how you decide where that is.
Classification is the throttle on governance cost. Set it honestly and your programme is sustainable; set it dishonestly and you are either broke or exposed.
The dimensions of impact
A good classification considers several dimensions together, because no single one captures risk on its own.
Severity of harm
The first question is how badly a wrong decision hurts someone. Denying a person urgent medical care, wrongly flagging them as a criminal, or cutting off their access to essential financial services sits at the severe end. Mis-ranking a product recommendation sits at the trivial end. Severity is the dominant dimension; a system capable of grave harm is high-risk almost regardless of its other properties.
Scale and reach
A decision that affects a handful of people is different from one applied to millions. Scale amplifies harm: a small bias that is statistically minor becomes a major societal harm when applied at population scale, and a single faulty rule can wrong an entire class of people before anyone notices. Reach also matters legally, because mass harms attract collective redress.
Reversibility
Can a wrong decision be undone, and how easily? A flagged transaction that a human can quickly clear is more forgiving than an automated account closure that destroys a small business's cash flow before anyone can intervene. The harder a decision is to reverse, the more rigour it warrants — and the stronger the case for a human checkpoint in front of it.
Autonomy
How much human judgement sits between the model's output and the real-world action? A model that merely informs a human who decides is lower-risk than one that acts directly with no review. Autonomy concentrates risk, because there is no second pair of eyes to catch an error before it becomes a harm. This dimension becomes central when we reach agentic systems later in the course.
Vulnerability of those affected
Decisions about vulnerable populations — children, the elderly, the financially distressed, people in medical crisis — carry heightened risk and heightened regulatory attention. The same system applied to a vulnerable group sits a tier higher than when applied to sophisticated, well-resourced parties who can absorb and contest an error.
Contestability and opacity
Finally, how easily can an affected person understand and challenge a decision? A decision they cannot see, understand, or appeal is riskier than one accompanied by clear reasons and a real route to redress. Opacity is itself a risk factor, not merely a compliance gap.
A workable tiering scheme
Most institutions converge on three or four tiers. The exact labels matter less than the discipline of differentiation. A representative scheme:
- Prohibited. Uses your organisation will not pursue at all — either because the law forbids them or because they fall outside your risk appetite. Naming these explicitly is valuable: it stops debates before they start.
- High-risk. Systems that make or heavily influence consequential decisions about people. These get the full apparatus: independent validation, documented human oversight, intensive monitoring, formal sign-off, and complete audit trails. Most of this course describes how to govern this tier.
- Limited-risk. Systems with modest impact or strong human mediation. These get proportionate controls — basic validation, transparency, and lighter monitoring — without the full machinery.
- Minimal-risk. Low-stakes systems where the cost of heavy governance would exceed any plausible harm. These get baseline hygiene — an inventory entry, basic testing — and otherwise proceed freely.
The point of the scheme is not bureaucratic neatness; it is to make the level of effort a deliberate, documented choice rather than an accident of whoever happened to build the system.
Common classification traps
Classification goes wrong in predictable ways. Knowing the traps is half the defence.
- Classifying the technology, not the use. "It's just a linear model, so it's low-risk." The model's mechanism is irrelevant; what matters is the decision it drives. A simple model denying mortgages is high-risk; a deep neural network suggesting playlists is not.
- Anchoring on intended use and ignoring foreseeable misuse. A system classified for a benign purpose can be repurposed, and regulators increasingly expect you to consider reasonably foreseeable misuse, not just the happy path.
- Salami-slicing. Breaking a high-risk system into components, each individually "low-risk", to dodge the tier. Examiners see through this immediately; classify at the level of the decision and its consequence.
- Optimistic self-assessment. The team that built the system is rarely the right judge of its risk. Classification should involve, or be reviewed by, an independent risk function with no stake in a lighter outcome.
- Set-and-forget. A system's risk profile changes as its use expands or its data shifts. A model approved for a narrow pilot and quietly scaled to a core decision has effectively changed tiers without anyone re-classifying it.
Documenting the classification
A classification is only as good as your ability to defend it, which means it must be written down with its reasoning. For each system, record what it does, the decision it drives, its score against each impact dimension, the tier you assigned, and — crucially — why. The "why" is what an examiner will probe. A classification that simply asserts "medium-risk" with no rationale is worthless under challenge; one that walks through severity, scale, reversibility, and autonomy and reaches a reasoned conclusion is defensible even if the examiner would have drawn the line slightly differently.
This record lives in your model inventory, the subject of later parts on documentation. It should be revisited on a schedule and whenever the system's use materially changes.
When in doubt, tier up
Classification involves judgement, and reasonable people will sometimes disagree about borderline cases. The safe default is to err upward. Over-governing a limited-risk system wastes some resources; under-governing a high-risk one is how firms end up explaining themselves to a regulator after a harm has already occurred. The asymmetry of consequences argues for caution at the margin. As you gain evidence that a system is behaving safely, you can revisit and, if justified, tier down — a decision that is itself documented and reviewed.
Worked examples across the tiers
Classification becomes clearer with examples that show the dimensions interacting rather than in isolation.
- A model recommending internal knowledge-base articles to support staff. Low severity, mediated entirely by a human who chooses whether to use the suggestion, fully reversible. Minimal-risk: an inventory entry and basic testing suffice.
- A model prioritising the order in which a support team handles tickets. Modest impact, but at scale it could systematically disadvantage some customers. Limited-risk: proportionate validation and periodic fairness checks, without the full apparatus.
- A model that automatically declines insurance claims below a threshold. High severity (denies people money they may be owed), large scale, hard to reverse from the claimant's perspective, and substantially autonomous. High-risk without question, regardless of how simple the model is.
- An agent that autonomously freezes accounts it judges fraudulent. Severe, irreversible in its immediate effect, high autonomy, and applied to people in financial distress. High-risk, and a candidate for mandatory human checkpoints before the irreversible action.
Notice that in none of these did the model's technical sophistication drive the tier. A trivial rule and a deep network land in the same tier when they drive the same decision — because, as Part 1 argued, the discipline is about the decision, not the mechanism.
The dynamic nature of risk
One of the most common ways classification fails is by treating it as a permanent label rather than a current assessment. A system's risk profile is dynamic, and several forces move it over time:
- Scope expansion. A model approved for a narrow pilot is quietly extended to a core, high-volume decision. Its tier should have risen with its reach, but nobody re-classified it.
- Population shift. A system extended to a more vulnerable population — say, from sophisticated commercial clients to retail consumers — takes on heightened risk that its original classification never contemplated.
- Increasing autonomy. A system that began as a human-mediated recommendation is gradually trusted to act on its own as confidence grows, concentrating risk that the original tier did not reflect.
- Aggregation. A model that was low-risk in isolation becomes consequential when its outputs feed a chain of other automated decisions, amplifying a small error into a large one.
The defence is to tie re-classification to change management (a later part): any material change in use, population, autonomy, or scale triggers a fresh look at the tier. A system whose classification has not been revisited since launch, but whose use has grown, is almost certainly mis-tiered.
Classification as a conversation, not a calculation
It is tempting to want a formula — score each dimension, sum, read off the tier. Resist the false precision. Classification involves genuine judgement, especially at the boundaries between tiers, and the value lies as much in the structured conversation as in the answer. Bringing the builders, the risk function, and the business owner together to reason through severity, scale, reversibility, autonomy, and vulnerability surfaces assumptions and disagreements that a formula would hide. The documented rationale that results — the account of why this system sits in this tier — is what makes the classification defensible and what a skilled examiner will actually probe. A number with no reasoning is indefensible; a reasoned judgement, even a debatable one, demonstrates exactly the deliberate risk awareness the whole discipline is built to produce.
The goal of classification is not a perfect number. It is a defensible, documented judgement about how much this system could hurt people — and a level of effort that honestly reflects it.
In the next part: governance foundations — once a system is classified, who owns it, who is accountable for its decisions, and how to structure roles so that accountability is real rather than diffuse.