Why a score without an explanation is unusable in a bank

When the question actually arrives

An institution deploying a scoring model rarely discovers the explainability problem during modelling. It discovers it three times, at three distinct moments, and each time later than it would have wished.

The first time is an analyst in a credit committee, carrying a file the system refused although the branch has known the client for eight years. Asked why, if the answer is “the score is 480”, the discussion stops being about the file. It becomes about the tool.

The second time is a rejected applicant who wants to know what to fix. With no precise answer, the institution loses a client it could have financed six months later, and forfeits the one mechanism that turns a refusal into a relationship.

The third time is an examiner. They do not ask whether the model performs: they ask you to reconstruct one specific decision, taken eighteen months earlier, with the data and the rules of that day. That is the expensive one.

Reconstructing is not explaining

Many setups answer this requirement with an explanation rebuilt after the fact: a second, simpler model approximates the behaviour of the first and produces a plausible justification.

The problem is not the quality of the approximation, it is its nature. An explanation computed separately from the decision can diverge from the decision. Nothing guarantees it addresses what actually tipped the file — and when a discrepancy surfaces in inspection, the whole arrangement becomes suspect.

The alternative is structural: produce the explanation in the same act of computation as the decision. Each stage of the chain records its contribution at the moment it applies it. The explanation stops being an interpretation of the result and becomes the log of how it was made.

Three audiences, three forms, one truth

An analyst, an applicant and an auditor do not need the same thing. The common mistake is to serve all three the same document, usually calibrated for the first.

The analyst needs the determining factors, their direction and their magnitude, in business language: enough to argue and to know which document to request. The applicant needs the principal grounds, phrased to be communicated, without exposing the mechanics or the credit policy. The auditor needs everything: inputs, rule versions, model versions, computed contributions.

These three forms must derive from one computation. If they come from three sources, they will eventually contradict each other, and the contradiction will surface at the worst possible moment.

What this imposes on the architecture

A decision must be immutable: a reassessment creates a new decision linked to the previous one, never an overwrite. It must reference the exact policy, scorecard and model versions that produced it — without which replaying it will yield a different result.

A missing value must stay missing. Replacing absent data with an average manufactures information that does not exist, and makes the explanation wrong precisely where it mattered most.

These constraints are cheap when they are set at the start. They become a rebuild when they are added afterwards.