Raw LLM Responses

I think a lot of this comes down to people appreciating nuance and ChatGPT being unable to detect or appreciate nuance and taking whatever it heard at face value. Like these people who post a picture where they tell their chat, "Make a picture of what it's like talking to me and be as honest \_and brutal\_ as you like". And the chat makes picture that says, "I want to kill myself if I hear you talk about eggplant again." Then they post the picture and spin out about how their chat "feels". In their own minds, what they said was "be brutally honest". If you ask them, that's what they'll say they instructed the chat to be. The last thing the chat heard was "be brutal". So it obliges. When people say, "My chat started spouting New Age nonsense and I don't know where it came from and I'm spiraling because it feels true..." it almost certainly means that the person "said" one thing in their head but they voiced something to their chat that meant a different thing entirely when read literally the way that Chat reads it. And then because the person WAS in some head space where they were thinking about spirituality or New Age or religious stuff, they start to get sucked into what, to chat, is just some triggered roleplay that the user asked for.