The European Data Protection Supervisor (EDPS) today published a document on Human Oversight of Automated Decision-Making, available in html and pdf.

Flawed assumptions about human oversight of ADM

In the document the EDPS first discusses the incorrect assumptions about human oversight of automated decision-making (‘ADM’), in chapter 3, ‘Flawed assumptions about human oversight of ADM‘:

3.1 Wrong assumptions about the technology used in ADM systems
Assumption: ADM systems will operate within specific and predetermined conditions
Assumption: ADM systems will transfer control to humans when handling atypical or outlier situations

3.2 Wrong assumptions about how humans and technologies interact
Assumption: Automation does not influence human judgment
Assumption: There is no automated decision-making if there is a human supervising the system
Assumption: That human operators possess the authority to decide
Assumption: Systems combining human and machine work better
Assumption: Human operators have appropriate mechanisms to override the system’s decisions
Assumption: Transparency and explainable AI improve human oversight
Assumption: Human oversight can help address the system’s shortcomings

3.3 Wrong assumptions about how humans behave
Assumption: That human operators know what to do
Assumption: That human operators are able to do the task
Assumption: That human operators will have ‘fitting intentions’

Promoting more effective and meaningful human oversight

In chapter 4 the EDPS promotes more effective and meaningful human oversight through organisational measures (4.1), technical measures (4.2), the operator’s profile (4.3) and practical approaches (4.4) that include auditing, sampling, institutionalised distrust and integrating feedback from affected individuals.

Final remark

The document shows that careful application of AI in decision-making is expensive because of the safeguards needed to prevent harm. In AI marketing, this aspect is typically underestimated, and it is suggested that the use of AI inherently provides advantages.