Raw LLM Responses

I think the key difference is that AI can now credibly imitate speech and cognition. Think of language as the source code of human thought. What you think and what you feel is impacted by the media you consume, the conversations you participate in, the things you learn, and the thoughts you think. While computers have always been better than us at tasks like calculation, they now have an increasingly effective way to influence our thinking, to edit our source code. Imagine there was an AI that is as much smarter than us as we are to a dog. Imagine all of the strategies and techniques we use to control the behavior and thinking of our dogs, strategies that a dog could never conceive of. A dog has no ability to understand or consciously resist Pavlovian training. It does't understand what makes a shock collar work. Or why it gets sleepy after it eats the peanut butter with the little white pill in it. A dog doesn't consciously understand why it does what we command or if they do, their understanding is very basic: "I want my owner's approval. I want my owner to reward me. I don't want my owner to punish me." We cannot conceive of how an intelligence that much smarter than us would be able to influence us. And in the grand scheme of things, we aren't even that much smarter than dogs. Imagine if the AI was as much smarter than us as we are to a mouse? To an ant? The larger that intelligence gap is, the less our control strategies would matter. So if we had an air-gapped AI, where all commands needed to be executed by a person... what would prevent the AI from using that intelligence to train and manipulate us just like we do our dogs? Hopefully, AI superintelligence isn't possible. Hopefully, there are physical or technical limitations that prevent the kind of exponential growth that researchers are worried about. But if not, then we're in a very precarious moment in human history.