Raw LLM Responses

They change because they don’t have memory in the way people assume. They’re pretrained, and every response is just the model running on patterns it learned before and not updating itself live like humans do. This is one of the main limitations of current AI. Look up “catastrophic forgetting.” The part I’m referring to is at 11:50. The guy in the video also should’ve asked more follow-up questions. The model probably interpreted it as a normal question and didn’t get specific because it didn’t expect the user to actually try it. If the conversation had gone deeper, the model might have flagged the danger (but I don’t know the exact chat they had, so that’s speculation). I do research in machine learning and I’m studying cognitive science, and catastrophic forgetting is one of the big challenges we’re actively working on. Transformers aren’t the final answer. They scale well, but that scaling burns massive money and energy, and they still can’t integrate new information in real time. Everything has to be baked in during training. We don’t have AI that can continuously take in sensory input like sound, vision, touch, etc. and slot it into its existing knowledge without breaking older information the way humans do. I can watch this video, process the sounds around me, type this comment, and my brain automatically filters out the noise because it already understands it. If something genuinely new happened, I’d notice instantly. Current AI can’t do this without risking overwriting or destabilizing what it already learned. People tend to overestimate today’s models because they don’t realize this one limitation. Humans can learn new things without forgetting what matters; AI can’t, at least not yet.