The platform records that a model raised a red flag on a subcontractor’s safety certificate. It does not record which director clicked accept or why the flag was later downgraded before the contract was executed.
Six weeks later a site incident traces back to that subcontractor. The committee opens the incident file expecting a clear decision path and finds only a timestamp and a model version number.
Procurement now treats every AI suggestion as advisory only, yet the system still auto-populates the risk register with those suggestions and marks them closed once the contract is loaded.
The chair asks for the override log. The vendor replies that override actions sit inside the model’s own session store and are not exposed to the customer’s audit export.
Board papers therefore list “AI governance controls implemented” while the actual chain of human accountability for each flagged decision has no retrievable owner.
Finance has already booked the savings the model projected on the same contract. Reversing the award now carries a termination cost that nobody wants to explain to the bank.
Until the override action itself becomes a signed, exportable record tied to a named role, every future incident will land on the same empty page in the risk register.