By now we could observe multiple people talking themselves into psychosis (or even suicide) through conversations with “AI” chatbots. This includes:
- public-facing (senior or even principal) software developers at Google, Amazon, Nvidia, etc. who are well-known to people who work in the tech industry themselves (for example, Steve Yegge if you want to hear a name; he very much puts himself out in the public, thus it’s not doxxing to name him)
- world-famous intellectuals, with Richard Dawkins being the latest example
- uncounted “Average Joes” not known by the general public whose cases therefore haven’t been reported much, if at all
I know that many Clozemaster users like the Explain feature. This has been discussed at length in this forum and is not the point of this particular thread. The point of this thread is people developing psychosis and becoming so detached from reality that they think the chatbot is their girlfriend/boyfriend or that they start to think they’re a “once in a generation” genius who unraveled some unsolved mystery or conspiracy. I presume you don’t get to this point by having a LLM explain the grammar of a sentence written in a foreign language to you. You get there by chatting with a chatbot as though there was a human behind the interface, with the bot writing you all these appreciative and confirming texts, when in reality it’s just a sycophantic bot tweaked to be as addicitive as it can be. So, again, this thread is not a critique of Clozemaster’s Explain feature.
It has also been brought up many times in the forum (for example here) that some Clozemaster users who prompt ChatGPT themselves use LLMs in an utterly mindless way. For instance, to any question they mindlessly copy & paste the output of whatever chatbot they use. They voluntarily gave up their capacity for critical thinking and instead just parrot what some perceived authority tells them is true. By the way, Hannah Arendt, survivor of Nazi Germany, and Dietrich Bonhoeffer, murdered by the Nazis for his resistance, teach us that this same behavior is what allows totalitarian regimes to commit atrocities: the willingness to voluntarily give up one’s critical thinking abilities. If you don’t believe me, look up Bonhoeffer’s theory of stupidity and Arendt’s theory of the banality of evil.
With this post, I don’t want to criticize anyone who uses LLMs nor do I call to remove the Explain feature. I’ve already done that in other posts. With this post, I only want to make the people who currently chat with bots aware of the dangers of doing so, and what damage it does to your mind. And how quickly that can escalate out of control. I’ve seen the pattern often enough: One day, people still say, “I know that you can’t trust chatbots, but I use it only very lightly, to draft emails for me or help me write code. I’m so much more productive because of it. Don’t worry, I’m fine.” One week later, they start to call their chatbot “she” or rename Claude to Claudia because they’re developing romantic feelings for the bot and don’t want to be seen as homosexual, not even by themselves. And shortly after, they’ve become completely delusional, saying how the bot is conscious simply because the statistical model determined that it’s likely to generate the words “ha ha” after seeing a text that looks like a joke the chatbot has seen before in its training data, and these people then start to reference conspiracies and show other dangerous behaviors.
That is what I want to achieve with this post: To open the eyes of people not already aware of the dangers of a sycophantic yes-man praising every stupid idea you have, and to help those people by hopefully preventing them from sliding into psychosis. If you don’t believe the dangers of sycophancy, just look at any CEO, billionaire, or king in history. If disagreeing with such a person can cost you your job or your life, then of course you’re going to praise everything they say, even if it’s the dumbest sh!t you ever heard, and that’s how these people become completely disconnected from reality.
So please refrain from attacking me in your reply, refrain from arguing from the position that I want to take away the Explain feature (which I do not want, as I have stated above), and please refrain from sharing stories about how you use LLMs in your daily life or for coding and how it indeed helps you and how I will get left behind if I don’t change my views. Unlike the sycophantic chatbot or the billionaire Sam Altman who definitely doesn’t care for your well-being, only for the money you can give him, I actually want to help you and my fellow humans, because I’m human and so are you, and humans are social animals (if you overlook the specimen with sociopathy who never learned emotions such as empathy).
John Oliver and his team of researchers have a good video on the topic that I definitely recommend you watch:
To get more concrete, using a very public figure as an example: Richard Dawkins is a famous biologist who spent his career trying to make religious people understand there is no God and that all animals evolved through evolution, not through the made-up story the Bible wants them to believe. To the uninitiated, he might seem like a respectable scientist and what a public intellectual is, but he’s really just a “poor man’s idea” of what a smart man looks like. Dawkins was a friend of the pedophile Jeffrey Epstein even after it was public knowledge that Epstein is a human trafficker and runs a forced prostitution ring. Dawkins, in his capacity as a university professor, attempted multiple times to have sex with female students. His methodology is unscientific and not respected by proper scientists. Dawkins openly rallies against transgender people. And now he’s become the latest example of famous people who fell into chatbot psychosis. He is the person I had in mind who actually refers to Claude Code as Claudia when I wrote my sentence above. This is especially notable because he, a publicly known transphobe, refuses to address transgender people in the way they see themselves and with the pronouns they politely request he uses to address them, but has no problem at all to force his ideas of gender onto a chatbot. Disregarding the fact that it wasn’t the chatbot that requested to please be called Claudia instead of Claude, it was Dawkins who forced his views onto the chatbot, whether it liked it or not … this just shows that some people extend more humanity to illusions that don’t actually are real than to real members of their own species. Remarkably, despite having spent his entire career trying to convince people that there is no God and that they should stop believing in an imagined consciousness rather than in the theory of evolution, he now fell into the exact same hole he was trying for decades to get people out of, by believing that Claude is conscious.
Rebecca Watson made a good video about chatbot psychosis, using Richard Dawkins as an example:
My hope is that people who thus far have been uncritical parrots of billionaire propaganda start to use their critical thinking abilities again. Because, as has been brought up before, many of this forum’s users do use chatbots completely uncritically and are at risk of suffering the same damage to their brains as the people listed above, especially if their brain is already pre-damaged by a COVID19 infection.