This report, written for the Anti-discrimination department of the Council of Europe, concerns discrimination caused by algorithmic decision-making and other types of artificial intelligence (AI). AI advances important goals, such as efficiency, health and economic growth but it can also have discriminatory effects, for instance when AI systems learn from biased human decisions. In the public and the private sector, organisations can take AI-driven decisions with far reaching effects for people. Public sector bodies can use AI for predictive policing for example, or for making decisions on eligibility for pension payments, housing assistance or unemployment benefits. In the private sector, AI can be used to select job applicants, and banks can use AI to decide whether to grant individual consumers credit and set interest rates for them. Moreover, many small decisions, taken together, can have large effects. By way of illustration, AI-driven price discrimination could lead to certain groups in society consistently paying more. The most relevant legal tools to mitigate the risks of AI-driven discrimination are nondiscrimination law and data protection law. If effectively enforced, both these legal tools could help to fight illegal discrimination. Council of Europe member States, human rights monitoring bodies, such as the European Commission against Racism and Intolerance, and Equality Bodies should aim for better enforcement of current nondiscrimination norms. But AI also opens the way for new types of unfair differentiation (some might say discrimination) that escape current laws. Most non-discrimination statutes apply only to discrimination on the basis of protected characteristics, such as skin colour. Such statutes do not apply if an AI system invents new classes, which do not correlate with protected characteristics, to differentiate between people. Such differentiation could still be unfair, however, for instance when it reinforces social inequality. We probably need additional regulation to protect fairness and human rights in the area of AI. But regulating AI in general is not the right approach, as the use of AI systems is too varied for one set of rules. In different sectors, different values are at stake, and different problems arise. Therefore, sector-specific rules should be considered. More research and debate are needed.
He was interviewed by Het Financieele Dagblad (in Dutch) about this study:
Computer doet niet aan gelijke behandeling van mensen: Raad van Europa waarschuwt: regels zijn nodig om discriminatie aan te pakken, Het Financieele Dagblad, 21 February 2019.
He was also interviewed on 22 February 2019 by the Council of Europe:
.@fborgesius, author of new #CoE report https://t.co/JHH2tNYb4F on #AI social consequences confirms #racism danger of #ArtificialIntelligence for predictive policing ahead of #AIFINCoE 26-27/2 #Helsinki ?? #HumanRights conf ‘Governing the #AI Game-Changer.’https://t.co/GMrizCQgwR pic.twitter.com/hubYL37DM8
— Council of Europe (@coe) 22 februari 2019