How Algorithmic Systems Automate Inequality

The deployment of predictive analytics in public administration is usually justified by the twin pillars of austerity and accuracy. Governments and private entities argue that automated decision-making systems reduce administrative bloat while eliminating the subjectivity of human caseworkers. For example, the Dutch government explicitly framed its automated childcare benefits fraud system as a means of replacing discretionary human judgment with objective, data-driven risk scoring. But the operational reality of these systems suggests a different outcome. Rather than removing bias, they frequently operationalize it, embedding historical inequities into the seemingly neutral architecture of code. For marginalized communities, the threat is not that the technology will fail, but that it will work exactly as designed, scaling up the systemic discrimination that policy is supposed to mitigate.

This dynamic is clearest in the digitization of the welfare state. When agencies turn to machine learning to detect fraud, they rarely begin with a blank slate, training their models on historical enforcement data. Because low-income and minority populations have historically been subject to higher rates of surveillance and policing, these datasets are saturated with selection bias. The algorithm, lacking sociopolitical context, interprets this over-representation as an objective indicator of risk, identifying correlation and deploying it as causality.