Raw LLM Responses

AI researcher here; I hate the generative AI bubble as much as anyone else, and I think AI safety is an incredibly important field, but I think there is a self-serving agenda being pushed by these authors. Yudkowsky has been ringing this bell for decades; he also has no formal education and has managed to make a name (and some money) for himself in AI spaces by making attention-grabbing statements like this. Nate seems like an educated computer scientist, but he is still personifying AI agents in a way that seems dangerous. AI doesn't lie, or deceive, or "try" to do anything, because it isn't capable of doing that; all it's doing is using very clever math to predict the most likely next word in a sequence, according to metrics and biases that are set by researchers and molded by the data we feed into them. Basically, stating that AI agents have super-human, or even human-like, intelligence is giving them far too much credit. MIRI pushes the agenda that AI have this super-human intelligence (or that it's always right around the corner) because it gives their research merit that otherwise doesn't exist. Bias exists, both in data and in researcher intent, and those biases present a problem when we use these agents for specialized tasks they weren't trained for, or accept their responses at face value. But claiming that these AI agents are more intelligent than us actually makes this more likely to happen, not less. Alignment happens when you learn what an AI agent was actually designed for, what use cases it is successful in, and using them for those cases only.