ACCA Strategic Professional · Strategic Business Leader · Machine learning, AI and robotics
Halden Logistics uses a deep neural network to set delivery prices. The board is told the model is highly accurate, but engineers cannot explain why it quotes particular prices, and a regulator has asked the company to justify individual quotes to customers. Which is the most appropriate governance response?
Halden should require explainability, using interpretable models or explanation tools, and retain human review of contested decisions. The problem is opacity against a duty to justify outcomes; high accuracy does not supply reasons, and the board remains accountable even if a vendor supplied the model.
- ARely on the model's overall accuracy, since accuracy satisfies any explanation requirement
- BRemove human oversight so that pricing is consistent and free of subjective influence
- CRequire explainability measures, such as interpretable models or documented explanation tools, and keep human review of contested decisionsCorrect
- DHand responsibility for the pricing outcomes to the software vendor through the contract
Explanation
The issue is the black-box nature of the model against a duty to justify decisions. Governance should secure explainability and human accountability for contested outcomes. Accuracy alone does not give explanations, and accountability cannot be wholly outsourced to a vendor.
Did you get it right without looking?
One question tells you little. A timed set on Machine learning, AI and robotics shows your real accuracy, how long you take and where you lose marks.
More Machine learning, AI and robotics questions
- Velmont Retail's RPA programme has delivered savings in the finance team, and the board now wants to extend it to customer correspondence, w…
- Kestrel Insurance wants a system that learns from 200,000 past claims, each labelled 'fraudulent' or 'genuine', so that it can classify new …
- Tessaly Insurance plans to replace most claims handlers with an AI system. The board is told the technology works, but staff fear job loss, …
- Marlowe Health uses a third-party AI tool to prioritise patients for treatment. An audit finds the tool performs poorly for a minority group…
- Corvane Bank deploys a deep neural network to approve or decline loans. A declined customer asks why, and the bank's staff cannot explain wh…
- Danvers Logistics has deployed RPA bots to reconcile supplier invoices. After the supplier portal's screen layout was redesigned, the bots b…