Raw LLM Responses

I asked the following question to ChatGPT: « According to you, is it possible to create a perfect safety for super intelligence, perpétuel safety as it keeps improving? If no why, if yes how. Answer for a non expert, then for an expert ». Here is his answer for a non-expert: Short answer: No, it’s not possible to create perfect, perpetual safety for a superintelligence that keeps improving itself. Why not? • Think of it like raising a child who never stops growing smarter, and who can re-write their own brain whenever they want. • You might teach them rules, but once they’re smarter than you and can change their own rules, you can’t be 100% sure they’ll always stay safe. • Computers are like that too: once they get the ability to change themselves and invent new strategies, you can’t prove they’ll always follow the “safe path” forever. So what can we do? • Instead of perfect safety, we aim for good enough safety: checks, limits, and “emergency stop” buttons that make it very unlikely something goes wrong. • It’s like building airplanes — they’re not perfectly safe, but layers of engineering and oversight make them extremely reliable. • With AI, the same idea applies: you combine safeguards, monitoring, and limits to reduce risks, even if perfection is impossible. For the experts, Short answer: No, perpetual perfect safety is formally impossible to guarantee for a recursively self-improving superintelligence. Hmmm 🤔