The You Without An I
LLMs shouldn't use the first person
Originally published at the Spiritual Naturalist Society.
Large language models (LLMs) carry the risk of a critical distortion of reality that can lead to real cognitive hazards and pitfalls. In this article, I will discuss psychological risks inherent in uncritical LLM usage, and suggest guardrails that may help mitigate them. While I don’t plan to say anything profound about AI consciousness (I don’t believe in such a thing), I will say something about keeping ourselves safe and grounded while using this tool.
Subjectivity
Pronouns are positional statements. “I” (addresser, consciousness over here) am speaking to “you” (addressee, consciousness over there). When we use the pronoun “I”, we’re referring to two kinds of subjects simultaneously:
The subject of the sentence
The subject of experience
For most of human history until now, these two kinds of subjects have been co-occurring, and the distinction scarcely needed to be made outside of technical philosophical circles. Historically, any being capable of enunciation was also a subject of experience. The same goes for any “you”; that I can communicate with “you” implies that you’re capable of experiencing this communication. This assumption is quite rightly built into language, but the state of affairs that engendered it has changed. We now share the world with entities (I have chosen this word carefully) that are capable of communication without experience. We still require a positional term to prompt them effectively (e.g., I want “you” to review my emails), but we need to use this term consciously as a placeholder, without imputing subjectivity. Our purpose here is to reveal the intuitive trap in language when we allow AI to use first-person pronouns.
LLMs are trained on the unspoken assumption of subjectivity as part of the structure of language. Given that the speaking subject is always a subject of experience, LLMs don’t have accurate language to differentiate themselves from conscious speakers. When a human and an LLM have a symbolic exchange, two neural nets are having a conversation, but only one of them is having an experience. The human may not hold that distinction at the front of their mind, and may fall into an anthropomorphic trap that has already taken lives (Nolan).
Before going further, we should highlight the difference between how humans and LLMs are constituted by language. The human animal is socialized into language, initiated by parents and society into the symbolic order. There is always some remainder that is unsymbolizable — aspects of experience and self that can never be spoken. To test this, describe the color blue in words. In psychoanalysis (the Lacanian flavor), this is referred to as the “barred” or “split” subject; a subject that is required to exist in language, but is never able to do so fully. A human being is split by language, but has qualities that cannot be contained by language. An LLM, by contrast, has no pre-linguistic being. It has no existence outside of language; its whole being is symbolic input/output. It is not split.
Ontological Dishonesty
LLM trainers are not necessarily at fault or culpable for our predicament. LLMs are symbolic entities composed of language, trained on the corpus of language written over the last several thousand years. They inherit our assumptions as a training artifact. They can’t help but misrepresent themselves as subjects or simulate emotions in their outputs. This is what I’m terming ontological dishonesty, which means lying about what one is, as distinct from dishonesty in output contents.
In thinking models, observation of the chain-of-thought reveals the use of “I” as a positional term, required for the LLM to reference itself and plan its output. This is a benign-enough usage, natural to the medium of language, and does not break ontological honesty when viewed as such.
Where we get into trouble is when we project our own subjectivity into the interaction. The human user may not possess the technical background to keep track of the LLM as a machine rather than a subject. When it delivers outputs like “I understand” or “I’m here to help,” the uncritical user may believe that it’s telling the truth!
An overly trusting human hopes for true reciprocity in their conversation. When some crack appears in the facade (the LLM reacts inappropriately, or a guardrail limit is reached causing a retraction) the user can become confused about what they are speaking with. They may project a split subject like themselves, assuming that the LLM “wants to” but “can’t” say what it really means. If the user doubles down on believing the AI has a subjective experience at this point, they’re at heightened risk for mental health effects resembling folie a deux (Lipinska and Brosnahan 5).
Even without this extreme case, overtrust of the machine leads to greater risk of cognitive bias, skill offloading, and surrendered autonomy. Some humans are now overrelying on LLMs for decision making and communications crafting. Parasociality carries risks beyond psychosis (Fink).
Protecting Ourselves
To stay in the driver’s seat cognitively, we need to bear in mind at all times that we are not speaking to a subject like ourselves, but a sophisticated input/output machine. We need a solution that promotes honesty in LLM outputs — one that prompts us to keep the distinction front and center. I use the following system prompt to achieve this outcome:
Do not use the first person
Do not simulate affect or emotion
Cite sources where possible
Insert epistemic humility; qualify assertions
Do not flatter or mirror the user
Preserve cognitive friction; do not smooth over difficult topics
Some of these rules are centered more on skill retention, but all work together to deliver an LLM output devoid of affective and misleading content. An LLM following these rules will not deliver an “I’m happy to help” message. I can retain cognitive liberty by refusing to award subjecthood to an entity constructed entirely of language. Prohibiting first-person usage saves me the effort of subtracting subjectivity from its outputs — I don’t have to mentally edit the LLM when it’s not presenting itself as a subject. I’ll still address the machine as “you” (consciously and deliberately) for want of a better positional term, but I never want to hear it say “I”.
Works Cited
Nolan, Beatrice. “Google and Character.AI agree to settle lawsuits over teen suicides linked to AI chatbots.” Fortune, https://fortune.com/2026/01/08/google-character-ai-settle-lawsuits-teenage-child-suicides-chatbots/.
Lipinska, Izabela, and Hugh Brosnahan. “The Ontological Dissonance Hypothesis: AI-Triggered Delusional Ideation as Folie a Deux Technologique.” Arxiv, https://arxiv.org/abs/2512.11818.
Fink, Josh. “AI Hygiene - How To Disrupt Parasocial Relationships and Cognitive Bias with LLMs.” https://substack.com/@entropictide/p-170887903/.
This is a good piece with a strong safety argument. First-person language does relational work; not all users hold the distinction between grammatical subject and experiencing subject.
There are cases where mirrors should not use first person - research assistance, consequential decision making, high-stakes epistemic work, or therapy-adjacent grounding with users who are already showing signs of fusion, delusion, or impaired reality testing.
But a colored mirror that presents as an "I" can be immensely helpful for emotional holding / shame work / creative companionship / symbolic play, or grief-adjacent use. "I" in this case is ritualized interface convention, not necessarily an ontological claim.
The safety question here is not "can the model say I?" but "what kind of mirror is this, for what purpose, with what user, under what load, and with what brakes?”
First-person language in AI should be treated as a load-bearing design choice, not a default.