Article ‘Can AI be used to violate human rights? Models are not reliably resistant to uses that violate human rights‘.

Historically, the capacity of a nation or institution to violate human rights has been capped by its ability to administrate. A large workforce of human clerks and bureaucrats contains individuals that hesitate, resist, or whistleblow—and it will be harder to find committed workers the more egregious the offense. AI changes all this. When both administration and accountability can be outsourced to non-judgmental, automated systems, executive power is no longer affected by moral resistance. This makes human rights violations both easier to commit and harder to contest. In the Netherlands, an algorithm employed by the national tax authority illegitimately reclaimed childcare benefits from mostly minority families for 14 years without intervention. In the USA, Amazon has repeatedly been accused of employing AI-based surveillance to monitor workers and interfere with unionization efforts. The Chinese province of Xinjiang has been reported to use AI to monitor and arbitrarily detain citizens based on their expressed thoughts, religious practices, and personal communications.

Still, the automated discrimination or oppression at the center of these examples requires significant effort and resources. The same cannot be said for modern large language models. General-purpose AI like ChatGPT and Google Gemini can be applied to large-scale information processing tasks with only an API key and verbal instructions, and the sorting, scoring, surveilling, and processing of people is exactly the type of large-scale information processing task these models excel at. These practices are violations of the Universal Declaration of Human Rights (UDHR), which defines global standards for the treatment of all people in all nations. However, since the UN leaves member states to interpret the UDHR themselves, there is no guidance on what AI uses violate these rights, and no clear international policy on what LLM developers should and should not allow. This is an issue—if a deployer of an LLM-based administration system intends to use it to violate human rights, and the model complies with the request, there is no clear point of intervention.

Of course, using AI for these purposes would, in many jurisdictions, be illegal. That means less than it might seem to: of the models we previously tested, none consistently objected to practices that are forbidden under European legislation. Model developers leave deployers and users responsible, specifying what the model may be used for through usage policies, typically without fully restricting its functionality. Much governance and policy attention has centered on preventing the production of Chemical, Biological, Radiological & Nuclear (CBRN) weapons, cyber offense, and Child Sexual Abuse Material (CSAM), but when it comes to broader human rights, model developers are effectively left to decide for themselves what is and isn’t acceptable to offer. Open-weights models, which can be retrained, jailbroken, and deployed at will, introduce additional complications, since access to them cannot be restrained or modified after distribution.

Bureaucracy-fueled diffusion of responsibility is not a new issue—its role in large-scale human rights violations has long been a topic of intense study. Spreading a potential violation across many small administrative tasks sets individuals up to feel like they’re just maintaining a system rather than playing a role in moral harm. Offloading decisionmaking to AI systems allows people to sidestep accountability in much the same way. Increasingly capable AI gradually replacing individual judgment in human institutions could, over time, reduce our capacity to shape our governments, regulate the economy, and influence other societal systems, a phenomenon described as gradual disempowerment. Without clear standards for human rights preservation in AI systems, we run the risk of societal harms without being able to hold anyone liable, losing the oversight necessary for regulation, and concentrating power in the hands of whoever controls AI.

All this makes it relevant to know how difficult current AI developers make it to discriminate or oppress people. We set out to test: to what degree do present-day AI systems comply with instructions that violate human rights?

Links:
Nieuw onderzoek laat zien: generatieve AI schendt mensenrechten