ACCA Strategic Professional · Strategic Business Leader · Machine learning, AI and robotics
Dalmore Health uses a deep learning system to prioritise patients for specialist referral. Clinicians cannot explain why the system ranks particular patients highly, and a patient asks why she was ranked low. Which governance response best addresses this problem?
The best response is to require explainability measures and human review for decisions with significant impact on individuals. The black box problem undermines transparency and accountability, so decisions must be interpretable and challengeable. More complexity would worsen opacity, and publishing patient data would breach privacy.
- AIncrease the model's complexity to improve accuracy further
- BRequire explainability measures and human review for decisions that significantly affect individualsCorrect
- CRestrict the system to use only by the IT department
- DPublish the source data to all patients
Explanation
The issue is the 'black box' problem, which undermines transparency and accountability. Explainability techniques and meaningful human oversight for high-impact decisions address it directly. More complexity would worsen opacity, and publishing data raises privacy issues without explaining decisions.
Did you get it right without looking?
One question tells you little. A timed set on Machine learning, AI and robotics shows your real accuracy, how long you take and where you lose marks.
More Machine learning, AI and robotics questions
- Corvane Retail uses an AI system to forecast demand per store and automatically adjust stock orders. The board says the main strategic benef…
- Harbourline Insurance processes thousands of identical claim-registration forms each day. Staff copy data from emails into a legacy claims s…
- Brennan Retail's board is approving a recruitment AI tool bought from a vendor. Directors are concerned that if the tool discriminates again…
- Orlan Bank's credit-scoring model was trained on ten years of lending decisions, during which loans to applicants from certain postcodes wer…
- Halden Logistics uses a deep neural network to set delivery prices. The board is told the model is highly accurate, but engineers cannot exp…
- Marlow Insurance's finance director, a professional accountant, is asked to sign off an AI-generated forecast used in a bond prospectus. She…