Amnesty International published a report on risk profiling, read the announcement: Risk profiling systems used to identify potential offenders are in breach of international law and must be banned – new report.
Their report is here: Automating suspicion | Risk profiling as a smokescreen for structural discrimination and inequality.

From the announcement:

The widespread use of risk profiling systems by public authorities in law enforcement, social security and migration is incompatible with international human rights law and must be banned, Amnesty International said in a new report.

Risk profiling is the assessment of whether a person or group is likely to break a law or a rule. Governments use it to identify potential offenders before any offence has been committed. These assessments are increasingly being automated based on artificial intelligence techniques and, in many cases, have led to human rights violations. This is the first time that risk profiling has been comprehensively assessed against international human rights standards.

 “This report shows how risk profiling systems affects people’s rights and offers scientific and legal arguments that human rights advocates, civil servants, oversight authorities, lawyers and judges can use to contest the use of these systems by states and other entities,

Alexander Laufer, Amnesty International Netherlands Researcher on technology and human rights.

Risk profiling systems are often cited as a method by which states can streamline services, improve cost-effectiveness, prevent crimes, including fraud, and control migration. These claims, based on the premise of scarce resources, have been debunked as a policy rationale, because they are empirically unsupported and politically useful assumptions that turn poverty and other social issues from political problems into an “efficiency” problem to be solved through automated rationing and surveillance.

Being targeted by risk profiling in high-stakes contexts like law enforcement, social security and immigration causes serious harms. People suffer psychological distress, stigmatization and false accusations of crime, potentially leading to homelessness, deportation, unfair denial of social benefits, or even imprisonment.

Amnesty’s findings are also relevant to the fight against money laundering and terrorist financing by companies, as required by the FATF and the EU.