How AI-Powered Emotional Surveillance Can Threaten Personal Autonomy and Democracy
Emotional surveillance applications powered by artificial intelligence (AI) technology have recently gained interest, with trials being conducted globally in areas from education and public services to border management. A few years back, brain-scanning headbands developed by an American start-up called Brainco were trialed on thousands of school children in China to detect those paying less attention, urging students to be attentive or at least “pretend” to be so. In the European Union, an AI border control system with emotion recognition capabilities called iBorderCtrl was tested in some EU countries to automate the border security process by working like a “lie-detector” for non-EU nationals who wanted to enter the EU. More recently, the transit agency Transport for London conducted a trial of emotional AI surveillance at certain stations of the London Underground with the claimed objective of preventing crimes, measuring satisfaction, and maximizing revenue by attracting more advertisers.
While the privacy implications of emotion recognition technology and its applications for surveillance have been widely debated, the potential impact of integrating AI-powered emotion recognition into surveillance practices on personal autonomy and democracy has received too little attention. Emotional AI surveillance is silently brewing behind the curtains and is signaling a transformative change coming soon to make things personal for each one of us. With the possibility of emotional AI surveillance becoming mainstream in the not-so-distant future hanging over our heads, a discussion on how emotional AI surveillance can threaten our autonomy and democracy as a society is long overdue.
What emotional AI surveillance systems can do goes beyond tracking and identifying people. They are more than facial recognition systems. They can sneak into your mind by reading your emotions from your facial expressions, voice, physiological signs, and words and expose it to those in power to control or manipulate you. What is at stake is more than your privacy. It is your mind, what is inside of it, and your agency to form your own authentic decisions and actions. Even more worrying is that these systems are not always accurate or free from bias, which means that you could face certain adverse implications also due to technical deficiencies in the system.
Related Knowledge Center: Artificial intelligence
On top of a failing business model, newsrooms must navigate the powerful — and not fully reliable — new capabilities of artificial intelligence and the deployment of large language models.
State of the business model | Copyright and intellectual property
Related Knowledge Center: Digital privacy and data security
If you’re online, you’re generating a constant stream of information about yourself — sometimes voluntarily, sometimes unwittingly — that is assembled and used to craft your online experience. And generate wealth for data brokers. Meanwhile, designers and managers of the digital infrastructure on which the world depends are engaged in a never-ending effort to thwart attempts, by both criminal enterprises and state actors, to exploit vulnerabilities in those systems.
Signalgate | State surveillance
Related Knowledge Center: State surveillance
While government monitoring of personal digital communications has traditionally been associated with authoritarian regimes such as China, the Trump regime has moved to use citizens’ personal information in unprecedented ways.
Digital privacy and data security | FISA — Section 702