Op 9 december jl. maakte de Europese Raad bekend dat er overeenstemming is bereikt met het Europees Parlement over de ‘AI Act’, de Europese verordening die het gebruik en het ontwikkelen van kunstmatige intelligentie, ‘artificial intelligence’, moet regelen. De Europese Commissie liet weten er verheugd over te zijn.

Intussen reageren burgerrechtenorganisaties kritisch.

Amnesty International kwam op 9 december met het bericht dat Europa heeft besloten om publieke massasurveillance niet te verbieden in de AI-wet en dat dit een wereldwijd negatief precedent oplevert. De Europese privacy koepel EDRi liet weten dat het te vroeg is om te juichen. Van Bits of Freedom kwam het bericht ‘Er is een akkoord bereikt over de Europese wet waarin kunstmatige intelligentie wordt gereguleerd (de zogeheten AI-verordening). We zijn sceptisch over de uitkomsten.‘

AI Act en de geprivatiseerde misdaadbestrijding
Intussen kom ik nergens tegen welke gevolgen de AI Act voor de privatisering van de misdaadbestrijding (‘witwasbestrijding’) heeft, terwijl de grote ondernemingen die misdaad moeten bestrijden, zoals banken en grote accountantsorganisaties, op grote schaal kunstmatige intelligentie inzetten. Het kabinetsvoorstel voor het bancaire sleepnet, dat nu in de koelkast van de Tweede Kamer staat, is gebaseerd op AI.

Het zal me benieuwen of de rechten van burgers in dit domein worden beschermd.

Meer informatie:

Europese instanties:

• Persbericht Europese Raad 9 december 2023.
• Persbericht Europese Commissie 11 december 2023.
• Bericht Europees Parlement van juni 2023, iets recenters zag ik nog niet.

Europese achtergrond:

• Commissievoorstel 14 april 2021.
• Standpunt Europese Raad 6 december 2022.
• Europese Commissie over AI, pagina.
• Europese Raad over AI, pagina; pagina over de digitale toekomst van Europa; pagina over de voordelen van digitalisering.

Burgerrechtenorganisaties:

• Nieuwsbericht Amnesty International: EU: Bloc’s decision to not ban public mass surveillance in AI Act sets a devastating global precedent.
• Bericht Bits of Freedom: Er is een akkoord bereikt over de Europese wet waarin kunstmatige intelligentie wordt gereguleerd (de zogeheten AI-verordening). We zijn sceptisch over de uitkomsten.
• Bericht EDRi: EU AI Act: Deal reached, but too soon to celebrate.

Aanvullingen

Aanvulling 14 december 2023
Lees over het gebruik van AI in de misdaadbestrijding onder meer:

• Ella Jakubowska: EU governments’ hypocrisy is on full display over dangerous police AI.
• France’s Digital Inquisition: Taking apart the secretive fraud detection algorithm that scores half of France’s population but pursues the most vulnerable, Lighthouse Reports.

Aanvulling 8 maart 2024
De denktank van het Europees Parlement heeft een overzicht van publicaties over artificial intelligence.

Lees bij La Quadrature du Net over de AI Act het artikel EU AI Act will fail commitment to ban biometric mass surveillance, met onder meer: “On 8 December 2023, EU lawmakers celebrated reaching a deal on the long-awaited Artificial Intelligence (AI) Act. Lead Parliamentarians reassured their colleagues that they had preserved strong protections for human rights, including ruling out biometric mass surveillance (BMS).
Yet despite the lawmakers’ bravado, the AI Act will not ban the vast majority of dangerous BMS practices. Instead, it will introduce – for the first time in the EU – conditions on how to use these systems. Members of the European Parliament (MEPs) and EU Member State ministers will vote on whether they accept the final deal in spring 2024.“.

Zij besluiten met: “Under this new law, we will be guilty by algorithm until proven innocent – and the EU will have rubber-stamped biometric mass surveillance. This will give carte blanche to EU countries to roll out more surveillance of our faces and bodies, which in turn will set a chilling global precedent.”

Aanvulling 20 maart 2024
Lees over artificial intelligence in de witwasbestrijding het artikel van Umut Turksen, Vladlena Benson en Bogdan Adamyk, allen verbonden aan het Europese financiële surveillance project TRACE: Legal implications of automated suspicious transaction monitoring: enhancing integrity of AI (pdf).
Opvallend is dat de auteurs alleen bezig zijn geweest met de financiële sector en de overheid. Voor de positie van de klanten van financiële instellingen was geen aandacht (anders dan de gebruikelijke verklaringen dat de grondrechten worden nageleefd). Zie het onderdeel Research design of the study, waarin staat: “To address our research questions, we applied doctrinal legal research by analysing law and court cases and a qualitative research approach by means of a series of semi-structured interviews. This research is conducted among banks’ senior managers, who are responsible for AI/ML systems in their banks“.

Aanvulling 25 juli 2024
The EU AI Act Newsletter meldt het artikel AI Act and its impacts on the European financial sector (EIOPA site) van Fausto Parente in het Eurofi Magazine. In de aankondiging staat onder meer:

The AI Act will impact the financial sector in a number of ways.

On the one hand, AI-based creditworthiness assessments by banks, as well as pricing and risk assessments in life and health insurance are considered high-risk AI use cases, and will therefore have to comply with heightened requirements for such AI applications.

These requirements are expected to be further developed by European standardisation bodies. Subsequently national competent authorities (NCAs) will need to ensure that financial institutions comply with the new AI governance and risk management requirements and standards, while assessing the extent to which more detailed sectoral guidance may be required for these AI use cases.

The AI Act will also introduce new requirements for so-called general purpose AI systems, including large language models and generative AI applications. Working closely with service providers such as Bigtechs, financial institutions are already experimenting with these new tools and assessing how they can take advantage of the significant opportunities they offer. The expectation is that these tools will become mainstream rather soon.

The European Commission’s new AI Office, which will be responsible for enforcing and overseeing the new rules for general-purpose AI systems, should ensure that service providers fulfil their responsibilities and assist users in implementing these systems. Under sectorial legislation financial institutions remain ultimately responsible for the tools and services they outsource. The oversight framework set out in the Digital Operational Resilience Act (DORA) for so-called “critical third-party service providers” could be useful here.

The remaining uses of AI in the financial services sector would largely be developed and used in accordance with existing legislation, without additional legal obligations arising from the AI Act. Given that the use of AI in use cases such as claims management, anti-money laundering or fraud detection in the financial services industry is already quite extensive, supervisors need to assess the extent to which existing rules are sufficient and where additional guidance may be needed for specific use cases. This would take into account considerations such as proportionality, fairness, explainability and accountability.

From another perspective, the European data strategy, which includes legislation such as the Data Act and the Data Governance Act, also plays a key role in shaping the landscape for the use of AI in the European financial sector. It facilitates the re-use of public sector databases or access to private datasets from connected devices, such as health wearables or connected cars, which could enable financial institutions to develop more innovative and tailored products and services, thereby making broadening competition in access to and use of data.

This is also the aim of the proposed Financial Data Access (FiDA) regulation, which will open consumers’ data held by financial institutions to third parties. In the insurance sector FIDA could facilitate the development of insurance dashboards, where consumers can access information about their insurance products from different providers on a single platform. This could potentially increase competition and enable consumers to make more informed choices.

Open questions remain about what data should be made available, how it is used, and on consumer protection. These will need to be addressed during the legislative process. Financial institutions should not be disadvantaged compared to non-financial ones, and consumer should always remain in effective control of where their data goes and how it is used.

NCAs will need to ensure that they integrate these new frameworks into their day-to-day supervisory activities. To this end, initiatives such as the ESA’s Digital Finance Supervisory Academy can bring economies of scale and support more agile up-skilling. In addition, NCAs should also progressively embrace the use of new technologies for supervisory purposes (Suptech), for instance deriving actionable insights from large datasets through AI.

Finally, it is also important to promote convergence, taking into account emergent European regulation, at the international level, as the International Association of Insurance Supervisors (IAIS) expects to do with the development of an AI application paper in the course of 2024.

Aanvulling 10 januari 2025
Zie mijn artikel Algoritmische opsporing door banken op grond van antiwitwasverordening niet gereguleerd in AI Act. Volgens het artikel dat ik bespreek is de AI Act niet op gebruik van AI in de witwasbestrijding door private ondernemingen van toepassing.
Op Verfassungsblog verscheen een artikel over het gebruik van AI door de overheid, Big Brother Is Analyzing You. Auswirkung der KI-Verordnung auf Verfahren der automatisierten Datenanalyse, 11 december 2024. Zie voorts KI im Einsatz für die Sicherheit en andere artikelen in de serie The EU AI Act’s Impact on Security Law