When the former chief scientist of OpenAI said it is the end of pre training as we know it after he left the company, he was most likely referring to GPT5, which was probably released as 4.5 to explain the modest gains in performance. Up until now, every major GPT release was an impressive leap forward and the recipe was something along the lines of 10x more compute and data. GPT4.5 signals the end for improvements purely from scaling data and compute during training.

Instead we are now at a phase where scaling happens when you use the model, with the model spending more time to “think” 🤔 or work out an answer and we are still exploring the limits of that. This is probably why it is a good time for OpenAI to consolidate all its offering into its next model and simply let the user adjust the thinking time they want, similar to what other providers are doing. Spending enough time on a problem can have significant impact as we have seen with the AI co scientist that was able to come up with a valid hypothesis in days that took scientist months.

At the same time, we are teaching those models how to use computer interfaces, instead of only being able to use natural language and some tools. This is still in its early days but once more mature will also be transformational since you will be able to instruct the AI to do complex workflows for you. In fact I do think that by the end of the year we should see actual glimpses of this 🍿