Manipulative phantoms in the machine: A legal examination of large language model hallucinations on human opinion formation

Aimen Taimur

doi:10.1007/978-3-031-91054-8_3

Manipulative phantoms in the machine: A legal examination of large language model hallucinations on human opinion formation

Aimen Taimur

Research output: Chapter in Book/Report/Conference proceeding › Chapter › Scientific › peer-review

Abstract

This paper investigates the novel implications of Large Language Model (LLM) hallucinations on cognitive liberty, the formation of informed opinions, and the potential for manipulative influence, especially in socio-psychological, academic and politically sensitive contexts. Employing a multidisciplinary methodology, the study integrates legal analysis to dissect the mechanisms driving LLM hallucinations. The analysis reveals the plausible risk for these hallucinations to distort public discourse, influence opinion formation, and propagate misinformation, thereby creating an unprecedented vulnerability in human-computer interactions. This study analyses existing legal frameworks, such as the EU AI Act, consumer protection law and Freedom of Thought, assessing their adequacy in addressing the manipulative impact of LLM hallucinations on independent human cognition.

Original language	English
Title of host publication	Privacy and identity management
Subtitle of host publication	Generating futures
Publisher	Springer Nature Link
Pages	59-77
Number of pages	19
Volume	705
ISBN (Electronic)	978-3-031-91054-8
ISBN (Print)	978-3-031-91053-1
DOIs	https://doi.org/10.1007/978-3-031-91054-8_3
Publication status	E-pub ahead of print - 22 May 2025

Publication series

Name	IFIP Advances in Information and Communication Technology (IFIPAICT, volume 705)

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

SDG 16 Peace, Justice and Strong Institutions

Keywords

LLM Hallucinations
Generative AI
Freedom of Thought
manipulation
Affective Computing

Access to Document

10.1007/978-3-031-91054-8_3

IFIP Summer School on Privacy and Identity Management, Madrid
Taimur, A. (Contributor)
2024
Activity: Participating in or organising an event types › Participation in workshop, seminar, course etc. › Professional

Cite this

Taimur, A. (2025). Manipulative phantoms in the machine: A legal examination of large language model hallucinations on human opinion formation. In Privacy and identity management: Generating futures (Vol. 705, pp. 59-77). (IFIP Advances in Information and Communication Technology (IFIPAICT, volume 705)). Springer Nature Link. Advance online publication. https://doi.org/10.1007/978-3-031-91054-8_3

@inbook{80c97d7cc97b4976bc4e19ec4a783494,

title = "Manipulative phantoms in the machine: A legal examination of large language model hallucinations on human opinion formation",

abstract = "This paper investigates the novel implications of Large Language Model (LLM) hallucinations on cognitive liberty, the formation of informed opinions, and the potential for manipulative influence, especially in socio-psychological, academic and politically sensitive contexts. Employing a multidisciplinary methodology, the study integrates legal analysis to dissect the mechanisms driving LLM hallucinations. The analysis reveals the plausible risk for these hallucinations to distort public discourse, influence opinion formation, and propagate misinformation, thereby creating an unprecedented vulnerability in human-computer interactions. This study analyses existing legal frameworks, such as the EU AI Act, consumer protection law and Freedom of Thought, assessing their adequacy in addressing the manipulative impact of LLM hallucinations on independent human cognition.",

keywords = "LLM Hallucinations, Generative AI, Freedom of Thought, manipulation, Affective Computing",

author = "Aimen Taimur",

year = "2025",

month = may,

day = "22",

doi = "10.1007/978-3-031-91054-8\_3",

language = "English",

isbn = "978-3-031-91053-1",

volume = "705",

series = "IFIP Advances in Information and Communication Technology (IFIPAICT, volume 705)",

publisher = "Springer Nature Link",

pages = "59--77",

booktitle = "Privacy and identity management",

}

Manipulative phantoms in the machine: A legal examination of large language model hallucinations on human opinion formation. / Taimur, Aimen.
Privacy and identity management: Generating futures. Vol. 705 Springer Nature Link, 2025. p. 59-77 (IFIP Advances in Information and Communication Technology (IFIPAICT, volume 705)).

Research output: Chapter in Book/Report/Conference proceeding › Chapter › Scientific › peer-review

TY - CHAP

T1 - Manipulative phantoms in the machine

T2 - A legal examination of large language model hallucinations on human opinion formation

AU - Taimur, Aimen

PY - 2025/5/22

Y1 - 2025/5/22

N2 - This paper investigates the novel implications of Large Language Model (LLM) hallucinations on cognitive liberty, the formation of informed opinions, and the potential for manipulative influence, especially in socio-psychological, academic and politically sensitive contexts. Employing a multidisciplinary methodology, the study integrates legal analysis to dissect the mechanisms driving LLM hallucinations. The analysis reveals the plausible risk for these hallucinations to distort public discourse, influence opinion formation, and propagate misinformation, thereby creating an unprecedented vulnerability in human-computer interactions. This study analyses existing legal frameworks, such as the EU AI Act, consumer protection law and Freedom of Thought, assessing their adequacy in addressing the manipulative impact of LLM hallucinations on independent human cognition.

AB - This paper investigates the novel implications of Large Language Model (LLM) hallucinations on cognitive liberty, the formation of informed opinions, and the potential for manipulative influence, especially in socio-psychological, academic and politically sensitive contexts. Employing a multidisciplinary methodology, the study integrates legal analysis to dissect the mechanisms driving LLM hallucinations. The analysis reveals the plausible risk for these hallucinations to distort public discourse, influence opinion formation, and propagate misinformation, thereby creating an unprecedented vulnerability in human-computer interactions. This study analyses existing legal frameworks, such as the EU AI Act, consumer protection law and Freedom of Thought, assessing their adequacy in addressing the manipulative impact of LLM hallucinations on independent human cognition.

KW - LLM Hallucinations

KW - Generative AI

KW - Freedom of Thought

KW - manipulation

KW - Affective Computing

U2 - 10.1007/978-3-031-91054-8_3

DO - 10.1007/978-3-031-91054-8_3

M3 - Chapter

SN - 978-3-031-91053-1

VL - 705

T3 - IFIP Advances in Information and Communication Technology (IFIPAICT, volume 705)

SP - 59

EP - 77

BT - Privacy and identity management

PB - Springer Nature Link

ER -

Taimur A. Manipulative phantoms in the machine: A legal examination of large language model hallucinations on human opinion formation. In Privacy and identity management: Generating futures. Vol. 705. Springer Nature Link. 2025. p. 59-77. (IFIP Advances in Information and Communication Technology (IFIPAICT, volume 705)). Epub 2025 May 22. doi: 10.1007/978-3-031-91054-8_3

Manipulative phantoms in the machine: A legal examination of large language model hallucinations on human opinion formation

Abstract

Publication series

UN SDGs

Keywords

Access to Document

Fingerprint

Activities

IFIP Summer School on Privacy and Identity Management, Madrid

Cite this