Manipulative phantoms in the machine: A legal examination of large language model hallucinations on human opinion formation

This paper investigates the novel implications of Large Language Model (LLM) hallucinations on cognitive liberty, the formation of informed opinions, and the potential for manipulative influence, especially in socio-psychological, academic and politically sensitive contexts. Employing a multidisciplinary methodology, the study integrates legal analysis to dissect the mechanisms driving LLM hallucinations. The analysis reveals the plausible risk for these hallucinations to distort public discourse, influence opinion formation, and propagate misinformation, thereby creating an unprecedented vulnerability in human-computer interactions. This study analyses existing legal frameworks, such as the EU AI Act, consumer protection law and Freedom of Thought, assessing their adequacy in addressing the manipulative impact of LLM hallucinations on independent human cognition.