Raw LLM Responses

I work at a well known large-ish tech company, and our top AI researcher gave an interesting presentation on the current state of LLMs. He described them as having two main parts: the pre-trained part and the “thinking” part. At this point, the pre-trained part is trained quite literally on the entirety of the internet, meaning that we’re probably close to an upper bound on the benefits we can get from that part. As he put it, how far LLMs can get in their capabilities depends on how AI companies can innovate on the “thinking” part. Admittedly, I’m not super knowledgeable in this area, so I wasn’t totally following, but I think this is where agentic AI comes in (specialized smaller models working together inside a bigger model). I think I agree with your assessment. It’ll be interesting to see if these models hit a hard upper bound in their capabilities.