> If GPT-6 was only larger, it would still only know the things in its training set, whereas a human can take arbitrary time to think, can go out and perform experiments, can Google/GPT things themselves, etc.

A GPT model already has many ways of thinking and self-distilling; for example, it can give itself a lot more time to think using 'inner monologue' techniques, which make it capable of calculating out step by step, using external tools like Python etc: https://www.gwern.net/docs/ai/gpt/inner-monologue/index A GPT model can also already Google things, see WebGPT https://openai.com/blog/webgpt/ (itself merely one entry in a rapidly-expanding area of retrieval models https://www.gwern.net/docs/ai/retrieval/index ). Enabling them to use web tools is likewise an active and increasingly successful area of research https://arxiv.org/abs/2202.08137#deepmind , with startups forming for that purpose: https://www.adept.ai/post/introducing-adept It can't 'run experiments', but the setting of reinforcement learning would allow that, and Decision Transformer (literally just a GPT trained on RL data and then given a prompt) is one of the more exciting directions - you may have seen Gato https://www.deepmind.com/publications/a-generalist-agent but have you also seen Multi-Game Transformer https://sites.google.com/view/multi-game-transformers ? Oh, that's just a single agent and you're not worried? Well that's fine because DT is so general it's almost trivial to make it do multi-agent RL as well https://arxiv.org/abs/2112.02845 https://arxiv.org/abs/2205.14953

Things are moving fast, and few people know all of the things that a GPT can do. (By the way, did you know that this 'big blob of inanimate knowledge' also knows what it doesn't know and can give well-calibrated predictions of accuracy https://arxiv.org/abs/2205.14334 ?)