Emotional AI and Deceptive Design: The Issue of Sovereignty for Subjectivity

Abstract

This article addresses the issue of digital sovereignty not only as a political and economic matter but also within the realm of the subject and subjectivity, or the human-machine relationship. We focus on two domains where the question of subject sovereignty is most explicitly raised: AI systems aimed at the automated inference of emotional and psychological characteristics, and the incorporation of dark design patterns inherited from behavioral economics in platform construction. In both cases, we analyze the implications for the question of subject sovereignty and the debate on AI regulation, guided by the central question: what kind of subject sovereignty could be proposed in AI platforms? We discuss how, in the attempt to impose barriers against manipulation or insidious influence on individuals and populations, regulatory frameworks risk asserting either a fully autonomous subject, free from influences, or a definitively vulnerable subject that needs protection. We point out that neither of these extremes holds up when it comes to understanding how subjectivities and psychological and emotional processes are engaged by AI platforms. Finally, we conclude that in the digital environments, platforms, and AI applications targeted in this article, subject or subjectivity sovereignty cannot be definitively ensured at the individual or legal level. Instead, it is a sociotechnical and technopolitical issue that must be continuously addressed and revised collectively.