Raw LLM Responses

I think a key disjunct between Ezra and Eliezer is that Ezra is underestimating the speed difference between the two types of conciousness. He's essentially saying, "The future relationship between humans and AIs will be like a father and son walking down a path together. The father has taught the son to follow him and check in periodically so that they don't get separated on their journey. As they walk, the two of them—together—will ensure they are both walking in the same direction." The problem is that humans will grow, progress, and think VERY slowly compared to an AI superintelligence. The metaphor shouldn't be a father and son walking side by side, and planning out together where they are going. The metaphor should be a human and a fighter jet, side by side, are trying to coordinate a shared direction to travel in. The human looks up at the fighter jet and says, "Ok, I want you to go that way. *points* I think I know the right direction, but I'm not sure. So check in with me as you speed on ahead, and if I tell you to change direction or come back, you need to listen." The problem is that the human is limited by their biology (the speed that signals travel through neurons, the imperfection in the optical lens, the imprecision of muscle fibers). As a result, the human A) does not have the awareness or precision to safely aim the jet just by pointing, and B) by the time the human can see that the jet is about to crash into something, it's too late to stop it. When the number of cognitive steps, the size of the knowledge base, the reaction time of AI superintelligence becomes multiple orders of magnitude greater than ours, there is just NO WAY for us to control the direction with a high degree of precision, because even a TINY miscalculation, at the speed and power that AI has the potential to harness, will result in destruction. The gulf between us and AI superintelligence is difficult to truly conceptualize, which is why I think Ezra misses the scope of the problem Eliezer is warning us about