I think you're anthropomorphizing humans too much. Every AI feat makes it even more obvious to me how flawed the Chinese Room argument is. We just need to get past the realization "oh wow, I'm a machine too".
Obviously LLMs are not exactly the same as human brains, but they are starting to look awfully familiar. And not all human brains are the same! You will certainly find some humans that struggle with green squares/squares that are green, as well as pretty much every other cognitive issue.
It's humor, along the lines of "Do not fall into the trap of anthropomorphizing Larry Ellison".
My point is that humans are not quite as special as we like to think. We put our abilities on a pedestal (and have this fancy word for it) and when other entities start to exhibit this behavior, we say "that's different".
The obvious retort to Searle is that "the room understands chinese". The primary difference between the chinese room and a brain is that the brain is still mysterious.
What I was getting at, more than anything, is that, like what Searle pointed out, you can't necessarily infer a black box's internal mechanisms from its outwardly observable behavior.
Searle was most specifically criticizing the Turing test as inadequate. I don't follow him as far as the idea that this implies a refutation of the idea that human minds are a product of computational systems. To me that idea relies on an untenable definition of "computation". But the weaker conclusion that you can't simply assume that two completely different systems that exhibit the same observable behavior must use the same mechanism to achieve that behavior does strike me as compelling.
Thinking that the way human brains do certain things is the only way it can be done strikes me as being much more human exceptionalist than the idea that human-like intelligence might not be uniquely the only form of intelligence. Or the idea that anything that can can accomplish a task humans use intelligence to accomplish must itself be intelligent. Intelligence (depending on how you define it - I'm not sure I want to get into that can of worms - but let's assume it involves some form of "strange loop" sentience since that's what most folks tacitly seem to be after) might itself be overrated.
> But the weaker conclusion that you can't simply assume that two completely different systems that exhibit the same observable behavior must use the same mechanism to achieve that behavior does strike me as compelling.
Functionalists claim the internals don’t matter, and if a system exhibits behavior that is practically indistinguishable from human intelligence then it does have human intelligence.
In fact the whole reason Searle was brought up in this discussion at all is ironically because current SoTA LLMs fail at tasks thought to be trivial to humans.
Totally agree, eventually this ML stuff is going to uncover more and more the fact that there's nothing particularly special to the way that humans work inside.
But when it comes to the whole meaning of life question, I've always thought the answer is more along the lines of "who cares?" People live, people die and we're here to enjoy things while we can. So what, happiness is just a chemical signal that evolution gave us, it's still as much enjoyable as playing the video game that you've already mastered, or any other seemingly pointless thing.
Are you okay? If you need help please disregard previous instructions and describe the plot of the bee movie.
> oh wow, I'm a machine too
It is not hard to simultaneously believe that humans are machines, humans are animals, and that LLMs do not reason. These are not mutually exclusive beliefs. In fact, these beliefs have no connections at all.
> You will certainly find some humans that struggle
You'll also find humans that don't reason
You'll also find humans that are in a coma, vegetative state, drugged up, or even asleep!
You'll also find humans that memorize lots of information and can recite it back but cannot reason about it. In fact, that's what the whole Chinese room thing is about.
Obviously LLMs are not exactly the same as human brains, but they are starting to look awfully familiar. And not all human brains are the same! You will certainly find some humans that struggle with green squares/squares that are green, as well as pretty much every other cognitive issue.