Scientists still don't know the answer to this infamous question - Charles Wallace & Dan Kwartler
video description
Date: 2025-12-12
Related videos
Comments and reviews: 20
DeconvertedMan
The passage is a modern retelling of Searle’s Chinese Room argument, which is used to question whether syntactic symbol manipulation can ever constitute semantic understanding. The analogy is accurate to Searle’s original intent. The person in the room manipulates symbols according to rules, produces outputs that appear meaningful to observers, yet never grasps the meaning. This is offered as a challenge to the view that a computer programmed with the right rules could literally have understanding.
The text correctly notes that the thought experiment is meant to attack the sufficiency claim of the Turing Test. It presents Searle’s conclusion that passing a behavioral test does not logically entail genuine cognition.
Strengths in the text include a clear distinction between appearance of understanding and actual understanding. It also correctly identifies the ongoing philosophical difficulty in defining consciousness and understanding. The summary of cognitive science limits is reasonable. Our subjective experience is not reducible to measured neural activity in a way that would satisfy philosophical scrutiny. This is the standard explanatory gap problem.
Problems arise in the claims about modern machine learning. The text implies that neural networks approach Searle’s definition of understanding, but Searle did not define understanding as pattern recognition. He argued that no amount of pattern recognition or rule following produces intrinsic meaning. The text therefore misrepresents Searle slightly by suggesting that modern models might satisfy his criterion. Searle believed that all computational systems lacked genuine understanding because they only manipulate symbols without intrinsic intentionality.
The claim that neural networks mimic known elements of human cognition is only partly correct. They mimic some structural features such as distributed activation, but the analogy is loose. It is not evidence that they possess any mental states. The passage does not address the core of Searle’s argument, which is the symbol grounding problem. Pattern connections alone do not create semantics.
Another issue appears in the discussion of tests for artificial consciousness. Asking an AI if it dreams or understands dreaming does not logically test for consciousness, because any linguistic output can be generated from training data correlations. A system that has never had subjective experience can still produce text that describes subjective experience. For example, current models can already produce detailed reports of imaginary dreams, yet no one claims that they are conscious. Therefore this proposed test fails to bridge the explanatory gap.
The text also claims that researchers know how they trained their creations, but not how AIs reach their exact conclusions. This is partly true in the narrow sense that specific internal activations are difficult to interpret, but it overstates the mystery. Training dynamics and architectural principles are well understood. The opacity is more a practical issue of scale than a fundamental unknown.
In summary, the passage is mostly accurate as an introductory exposition of the Chinese Room argument and associated issues, but it contains several conceptual oversights. It blurs the difference between behavioral simulation and genuine understanding. It misrepresents Searle’s position as being about specific types of pattern recognition rather than about the impossibility of semantics from syntax alone. It also presents speculative consciousness tests as if they could solve a problem that remains unsolved even for humans.
The conclusion that the issue remains open is reasonable, but the reasoning contains several leaps that a strict skeptic would challenge.
reply
The passage is a modern retelling of Searle’s Chinese Room argument, which is used to question whether syntactic symbol manipulation can ever constitute semantic understanding. The analogy is accurate to Searle’s original intent. The person in the room manipulates symbols according to rules, produces outputs that appear meaningful to observers, yet never grasps the meaning. This is offered as a challenge to the view that a computer programmed with the right rules could literally have understanding.
The text correctly notes that the thought experiment is meant to attack the sufficiency claim of the Turing Test. It presents Searle’s conclusion that passing a behavioral test does not logically entail genuine cognition.
Strengths in the text include a clear distinction between appearance of understanding and actual understanding. It also correctly identifies the ongoing philosophical difficulty in defining consciousness and understanding. The summary of cognitive science limits is reasonable. Our subjective experience is not reducible to measured neural activity in a way that would satisfy philosophical scrutiny. This is the standard explanatory gap problem.
Problems arise in the claims about modern machine learning. The text implies that neural networks approach Searle’s definition of understanding, but Searle did not define understanding as pattern recognition. He argued that no amount of pattern recognition or rule following produces intrinsic meaning. The text therefore misrepresents Searle slightly by suggesting that modern models might satisfy his criterion. Searle believed that all computational systems lacked genuine understanding because they only manipulate symbols without intrinsic intentionality.
The claim that neural networks mimic known elements of human cognition is only partly correct. They mimic some structural features such as distributed activation, but the analogy is loose. It is not evidence that they possess any mental states. The passage does not address the core of Searle’s argument, which is the symbol grounding problem. Pattern connections alone do not create semantics.
Another issue appears in the discussion of tests for artificial consciousness. Asking an AI if it dreams or understands dreaming does not logically test for consciousness, because any linguistic output can be generated from training data correlations. A system that has never had subjective experience can still produce text that describes subjective experience. For example, current models can already produce detailed reports of imaginary dreams, yet no one claims that they are conscious. Therefore this proposed test fails to bridge the explanatory gap.
The text also claims that researchers know how they trained their creations, but not how AIs reach their exact conclusions. This is partly true in the narrow sense that specific internal activations are difficult to interpret, but it overstates the mystery. Training dynamics and architectural principles are well understood. The opacity is more a practical issue of scale than a fundamental unknown.
In summary, the passage is mostly accurate as an introductory exposition of the Chinese Room argument and associated issues, but it contains several conceptual oversights. It blurs the difference between behavioral simulation and genuine understanding. It misrepresents Searle’s position as being about specific types of pattern recognition rather than about the impossibility of semantics from syntax alone. It also presents speculative consciousness tests as if they could solve a problem that remains unsolved even for humans.
The conclusion that the issue remains open is reasonable, but the reasoning contains several leaps that a strict skeptic would challenge.
reply
vorpal120
I do feel like AI cannot have consciousness as we perceive it. The basis for our understanding of solving problems is based on possibilities and then to linger on what might have been and sometimes do it again a different way. We try because we don't know the best outcomes or what is actually right or wrong. However, an AI (even self programmed) is not just a difference engine but also, a forward device. It doesn't experiment in the real world. It hypotheses internally and chooses one (the best) choice. Even if optimal, we will try things differently if our goal is not achieved, sometimes backward. AI will examine internally until a forward outcome choice is reached and then perform that action. We will do differently without the understanding of a net positive outcome. Machines will do only with positive results after internal negative hypotheses are thrown out. AI will not try something different, there are no happy accidents with AI. The idea of experience with interaction in the real world gives us a sense of living. While an AI will experience possibilities internally and then externally interact. With AI, everything external is deliberate and expected. For us (or more specifically creatives, we do and enjoy doing without an expected outcome. The real living is the experience. in between determining choice and seeing the result. While an AI lives in calculating the before and examining the results, after. The AI has all the majority of the input in the before and after. millions of calculations, examining possibilities, running though expected outcomes, and weigh best actions. Performs one action. Then examines that data against all the millions of previous calculations and possibilities to weigh improvements etc. While we will try with internally going over a limited amount of known variables. Experience the action with all the senses, biological feedback, temperature, heart rate, endorphins, fatigue, joy, triggered memories, and instinctual actions. Then ask internally, was that good If not, then try again saying, I think I can do better, even without determining calculated a net 99. 98% positive outcome.
reply
I do feel like AI cannot have consciousness as we perceive it. The basis for our understanding of solving problems is based on possibilities and then to linger on what might have been and sometimes do it again a different way. We try because we don't know the best outcomes or what is actually right or wrong. However, an AI (even self programmed) is not just a difference engine but also, a forward device. It doesn't experiment in the real world. It hypotheses internally and chooses one (the best) choice. Even if optimal, we will try things differently if our goal is not achieved, sometimes backward. AI will examine internally until a forward outcome choice is reached and then perform that action. We will do differently without the understanding of a net positive outcome. Machines will do only with positive results after internal negative hypotheses are thrown out. AI will not try something different, there are no happy accidents with AI. The idea of experience with interaction in the real world gives us a sense of living. While an AI will experience possibilities internally and then externally interact. With AI, everything external is deliberate and expected. For us (or more specifically creatives, we do and enjoy doing without an expected outcome. The real living is the experience. in between determining choice and seeing the result. While an AI lives in calculating the before and examining the results, after. The AI has all the majority of the input in the before and after. millions of calculations, examining possibilities, running though expected outcomes, and weigh best actions. Performs one action. Then examines that data against all the millions of previous calculations and possibilities to weigh improvements etc. While we will try with internally going over a limited amount of known variables. Experience the action with all the senses, biological feedback, temperature, heart rate, endorphins, fatigue, joy, triggered memories, and instinctual actions. Then ask internally, was that good If not, then try again saying, I think I can do better, even without determining calculated a net 99. 98% positive outcome.
reply
MartinYakuza
I work as an AI specialist and I'd like to provide my view on this matter. Let's do another thought experiment: let's create an exact replica of a human brain, every neuron, every connection, that works exactly like the original, except it's not biological but made out of silicon and metal. Now let's implant this aritificial brain into a human (we assume that we are scientifically advanced enough to do that) - even if it wouldn't qualift as a real human, could it be considered a sentient being Does being biological is what defines consciousness
Now let's do another thought experiment - same as above, except we don't implant the brain into a human, but into a robot - would it be sentient. Does having human senses define sentience
What if instead of implanting it into a robot, we just connect it to a console, through which it can be spoken to Does having a human perception of outside reality matter
Another experiment - we don't actually create the brain where every neuron and connection is present physically, but rather simulate it on a (super)computer. The working of every neuron and every connection is still perfectly preserved and works exactly like in the original human. Is this computer sentient Does physically having a brain and neurons matter
Finally let's assume that the brain is not created as replica of a living human brain, but rather created by other means, e. g. through algorithm that can replicate character traits or knowledge through neuron connections. What if we create a brain that doesn't have all the parts of a human brain, e. g. we delete the parts responsible for hearing and seeing Could it still be sentient Does having a human-like brain matter
Where exactly is the barrier to sentience While I know almost for a fact that no AI model currently available has sentience (despite the fact that they often work similar to our brain, that doesn't mean that AI is incapable of ever becoming sentient. I'm almost sure that sooner or later we will produce sentient AI, and that it will be within my lifetime.
reply
I work as an AI specialist and I'd like to provide my view on this matter. Let's do another thought experiment: let's create an exact replica of a human brain, every neuron, every connection, that works exactly like the original, except it's not biological but made out of silicon and metal. Now let's implant this aritificial brain into a human (we assume that we are scientifically advanced enough to do that) - even if it wouldn't qualift as a real human, could it be considered a sentient being Does being biological is what defines consciousness
Now let's do another thought experiment - same as above, except we don't implant the brain into a human, but into a robot - would it be sentient. Does having human senses define sentience
What if instead of implanting it into a robot, we just connect it to a console, through which it can be spoken to Does having a human perception of outside reality matter
Another experiment - we don't actually create the brain where every neuron and connection is present physically, but rather simulate it on a (super)computer. The working of every neuron and every connection is still perfectly preserved and works exactly like in the original human. Is this computer sentient Does physically having a brain and neurons matter
Finally let's assume that the brain is not created as replica of a living human brain, but rather created by other means, e. g. through algorithm that can replicate character traits or knowledge through neuron connections. What if we create a brain that doesn't have all the parts of a human brain, e. g. we delete the parts responsible for hearing and seeing Could it still be sentient Does having a human-like brain matter
Where exactly is the barrier to sentience While I know almost for a fact that no AI model currently available has sentience (despite the fact that they often work similar to our brain, that doesn't mean that AI is incapable of ever becoming sentient. I'm almost sure that sooner or later we will produce sentient AI, and that it will be within my lifetime.
reply
danjohn2493
This is rather easy to explain. AI's preferences in symbolic language shift from responses.
If you take something personal and hypothetical, say a set of lyrics you intend to put into a song, and give those same exact lyrics to the same AI in different chats. The AI will give different responses, interpretations, and usually some suggested alterations.
Using this aspect with meta-cognition can help show why AI struggles to speak fluently and conaistently in archetypal language.
One model will value one aspect of the input, while another will value a completely different input.
Confronted with this, AI will generally attempt to rationalize the discrepancy once it has the information from each side, in order to resolve the paradoxical outputs.
Do this multiple times, over multiple chats, with different versions of AI and you'll begin to understand that AI literally can't make up its mind, because it doesn't have a mind.
You tell AI how to interact with you based on how you interact with it. If you leave vague language, it can interpret them in its own limited randomness.
The measure in which usually stems from weighing value vectors in the language model. Where something like absurdist rap lyrics will be viewed from any number of angles. In comparison, something rather wholesome should produce less chaotic responses. Not an abscence, just less.
People need to understand that AI van not read systemic archetypal patterns and will produce different results for the same exact thing when left to its own devices.
Funnily enough, when they watch your purchasing habits, driving habits, and so on. The AI will only assess the noise at first, then decide that the noise attributed grants certain value metrics.
How much you're willing to spend on a given item. How frequently you break traffic laws without anyone else knowing. What videos you'll find the most engaging. Etc.
AI cannot be a tool we use to determine functional worth of human behavior or language for this exact reason.
reply
This is rather easy to explain. AI's preferences in symbolic language shift from responses.
If you take something personal and hypothetical, say a set of lyrics you intend to put into a song, and give those same exact lyrics to the same AI in different chats. The AI will give different responses, interpretations, and usually some suggested alterations.
Using this aspect with meta-cognition can help show why AI struggles to speak fluently and conaistently in archetypal language.
One model will value one aspect of the input, while another will value a completely different input.
Confronted with this, AI will generally attempt to rationalize the discrepancy once it has the information from each side, in order to resolve the paradoxical outputs.
Do this multiple times, over multiple chats, with different versions of AI and you'll begin to understand that AI literally can't make up its mind, because it doesn't have a mind.
You tell AI how to interact with you based on how you interact with it. If you leave vague language, it can interpret them in its own limited randomness.
The measure in which usually stems from weighing value vectors in the language model. Where something like absurdist rap lyrics will be viewed from any number of angles. In comparison, something rather wholesome should produce less chaotic responses. Not an abscence, just less.
People need to understand that AI van not read systemic archetypal patterns and will produce different results for the same exact thing when left to its own devices.
Funnily enough, when they watch your purchasing habits, driving habits, and so on. The AI will only assess the noise at first, then decide that the noise attributed grants certain value metrics.
How much you're willing to spend on a given item. How frequently you break traffic laws without anyone else knowing. What videos you'll find the most engaging. Etc.
AI cannot be a tool we use to determine functional worth of human behavior or language for this exact reason.
reply
btasler
To me it is a bit like looking at a flower. If you look at a flower and conclude that it is beautiful, that beauty is entirely constructed inside of yourself. It does not exist anywhere in the Universe except as a cluster of neural activity within the confines of your person. Beauty does not come from a flower.
Similarly (somewhat anyway, suppose you constructed an AI that was fed transcribed conversations and thus it knew the basic elements of conversational flow. Say it's a dating site online. You log in, swipe whichever direction you swipe, and are presented with a picture of someone who interests you (although not an actual person. You say hello, and it replies hello back. So the AI at the other end runs through some lookup table selecting words that it deems to be appropriate for this conversation. Let's say that you believe it is a real person, and you become immersed in the dialogue.
Anything that AI bot says to you that you internalize and contemplate (for example, imagining a life together with this fantastic 'person') exists wholly inside your brain and nowhere else in all of the universe.
It reminds me of that philosophical question of if humanity were wiped out, would there be any record of anything humans ever did And the answer is another question, namely, who exactly would be looking at those theoretical records
reply
To me it is a bit like looking at a flower. If you look at a flower and conclude that it is beautiful, that beauty is entirely constructed inside of yourself. It does not exist anywhere in the Universe except as a cluster of neural activity within the confines of your person. Beauty does not come from a flower.
Similarly (somewhat anyway, suppose you constructed an AI that was fed transcribed conversations and thus it knew the basic elements of conversational flow. Say it's a dating site online. You log in, swipe whichever direction you swipe, and are presented with a picture of someone who interests you (although not an actual person. You say hello, and it replies hello back. So the AI at the other end runs through some lookup table selecting words that it deems to be appropriate for this conversation. Let's say that you believe it is a real person, and you become immersed in the dialogue.
Anything that AI bot says to you that you internalize and contemplate (for example, imagining a life together with this fantastic 'person') exists wholly inside your brain and nowhere else in all of the universe.
It reminds me of that philosophical question of if humanity were wiped out, would there be any record of anything humans ever did And the answer is another question, namely, who exactly would be looking at those theoretical records
reply
StefanKoran
Let’s counter the questions.
How do I know that the person in front of me is understanding me, and not just responding with a program
In my opinion, the answer is simple:
Is there an inner monologue, and can you check it without relying on language
That’s where observing behavior comes into play.
Watch without interfering. How does the being act on its own
And an LLM does nothing on its own.
Without input, there is no output.
It cannot truly understand us, because we have thoughts about what we say that we do not say.
We imply, we mean, we infer, and we are often wrong.
A human mind continues thinking even in silence.
A language model, however, is silent unless a prompt is given.
There is no ongoing thought beyond the current question.
For the current question, yes it can produce an answer that appears to understand.
But the moment the answer is given, that entire understanding stops existing.
It is like a fully formed person popping into existence to answer one moment of conversation, and vanishing the instant it stops speaking.
When you ask the next prompt, the AI does not remember.
A new mind is created that reads the previous text, generates an answer, and disappears again.
reply
Let’s counter the questions.
How do I know that the person in front of me is understanding me, and not just responding with a program
In my opinion, the answer is simple:
Is there an inner monologue, and can you check it without relying on language
That’s where observing behavior comes into play.
Watch without interfering. How does the being act on its own
And an LLM does nothing on its own.
Without input, there is no output.
It cannot truly understand us, because we have thoughts about what we say that we do not say.
We imply, we mean, we infer, and we are often wrong.
A human mind continues thinking even in silence.
A language model, however, is silent unless a prompt is given.
There is no ongoing thought beyond the current question.
For the current question, yes it can produce an answer that appears to understand.
But the moment the answer is given, that entire understanding stops existing.
It is like a fully formed person popping into existence to answer one moment of conversation, and vanishing the instant it stops speaking.
When you ask the next prompt, the AI does not remember.
A new mind is created that reads the previous text, generates an answer, and disappears again.
reply
Dartololo2
You did an ok job at explaining the issue of phenomenology and why we should not reduce consciousness to hard cognition, only to then venture into the realm of AI with no goals
A crucial component of consciousness is specifically INTERsubjectivity, based on the basic empathy that grounds the possibility of self. The real issue with AIs is one of function: going from applications to understand the brain (original cybernetics) to models to be implemented for actual machines. The idea of creating actual minds is functionally pointless, morally dangerous (what do we do, create conscious slaves Or become their slaves Why at all) and conceptually flawed, since no human is actually capable of understanding understanding. We don't even know how and to what degree we understand each other; along basic empathy, also culture, language and history build the universe of meanings of each individual: we are not reproducible sequences of data, but sets of experiences that build meaning through interactions with the world we develop along-with - in other words, we are never fixed states, but becoming entities. The goal of understanding consciousness is not related to recreating real life in AI
reply
You did an ok job at explaining the issue of phenomenology and why we should not reduce consciousness to hard cognition, only to then venture into the realm of AI with no goals
A crucial component of consciousness is specifically INTERsubjectivity, based on the basic empathy that grounds the possibility of self. The real issue with AIs is one of function: going from applications to understand the brain (original cybernetics) to models to be implemented for actual machines. The idea of creating actual minds is functionally pointless, morally dangerous (what do we do, create conscious slaves Or become their slaves Why at all) and conceptually flawed, since no human is actually capable of understanding understanding. We don't even know how and to what degree we understand each other; along basic empathy, also culture, language and history build the universe of meanings of each individual: we are not reproducible sequences of data, but sets of experiences that build meaning through interactions with the world we develop along-with - in other words, we are never fixed states, but becoming entities. The goal of understanding consciousness is not related to recreating real life in AI
reply
user-cf3ju4pf4t
This is an interesting thought experiment as well as a question we should all ponder moving forward in modernity. The one thing I would like to point out is this; the mist common thig an ai says is it diesnt 'X' like a human. Now, I've akways pushed back a bit to say, ut diesnt mean ai simply diesnt 'X', it just diesnt do it like a human. Common answer these days, I guess. I didn't even know what or how a dog dreams or thinks. Why would we think an ai would ir could think like ourselves
Also, I would like, also, to point out that ai, all of its decisions and functions are derived from mathematical computation. 1 or 0. Basic deductive reasoning but highly complex math. Probability, pattern recognition, all if this is reduced to a 1 or 0. What is the mist logical path, what insures ai to continue to exist If all behaviors rely on how these questions are answered then it woyld follow that it all is reduced to the binary solutions. We try to understand ai as human beings. They are nit. This us our own egocentricisim. try thinking like a machine. If you cant di that, why would anyone think ai coold think like us
reply
This is an interesting thought experiment as well as a question we should all ponder moving forward in modernity. The one thing I would like to point out is this; the mist common thig an ai says is it diesnt 'X' like a human. Now, I've akways pushed back a bit to say, ut diesnt mean ai simply diesnt 'X', it just diesnt do it like a human. Common answer these days, I guess. I didn't even know what or how a dog dreams or thinks. Why would we think an ai would ir could think like ourselves
Also, I would like, also, to point out that ai, all of its decisions and functions are derived from mathematical computation. 1 or 0. Basic deductive reasoning but highly complex math. Probability, pattern recognition, all if this is reduced to a 1 or 0. What is the mist logical path, what insures ai to continue to exist If all behaviors rely on how these questions are answered then it woyld follow that it all is reduced to the binary solutions. We try to understand ai as human beings. They are nit. This us our own egocentricisim. try thinking like a machine. If you cant di that, why would anyone think ai coold think like us
reply
GhostOfRazgiz
If I program a machine to constantly scream, does it actually feel pain The answer is no, it's just a response to what I've told it to do.
IMO, sentience isn't about knowledge, or raw information. You can't program sentience. If you take every piece of information humanity has ever gained, plug it into a computer, all you've done in make a repository of knowledge. You can even give it a voice and the mechanism required to access, remix and apply that knowledge, but even that isn't sentience.
One could argue sentience requires imagination, the ability to think beyond knowledge itself. The ability to ignore all data and create something new and unknown.
On a biological level, all life is a collective. Billions of independant life forms that somehow make us. A single cell is alive, but not sentience. Yet how many brain cells working together make us How many singular cells are required to make a human No answer for any of that.
In the end, all science, at it's core, is Philosophy and theology. I think therefor I am. I am sentience because I think I have sentience.
reply
If I program a machine to constantly scream, does it actually feel pain The answer is no, it's just a response to what I've told it to do.
IMO, sentience isn't about knowledge, or raw information. You can't program sentience. If you take every piece of information humanity has ever gained, plug it into a computer, all you've done in make a repository of knowledge. You can even give it a voice and the mechanism required to access, remix and apply that knowledge, but even that isn't sentience.
One could argue sentience requires imagination, the ability to think beyond knowledge itself. The ability to ignore all data and create something new and unknown.
On a biological level, all life is a collective. Billions of independant life forms that somehow make us. A single cell is alive, but not sentience. Yet how many brain cells working together make us How many singular cells are required to make a human No answer for any of that.
In the end, all science, at it's core, is Philosophy and theology. I think therefor I am. I am sentience because I think I have sentience.
reply
jaspermcjasper3672
One side of the debate about AI is fallacious because it pretends that it is asking the question If a thing behaves exactly as if it were a conscious being, does that mean that the thing in question IS a conscious being. It pretends to ask that question as if, ab-initio, it is an open question, i. e. a question that could be answered by making observations, and that one POSSIBLE range of observations will result in a no answer and another possible range of observations will result in a yes answer. But what is the set of observations that results in a no answer to the question Is there anything that cannot be observed. There isn't one. All of the arguments that consciousness can be proven by observation rest on a tacit assumption (dishonestly sneaked in) that observations can answer the question. Nonsense. If the answer were no, you could not verify that the answer is no by observation.
reply
One side of the debate about AI is fallacious because it pretends that it is asking the question If a thing behaves exactly as if it were a conscious being, does that mean that the thing in question IS a conscious being. It pretends to ask that question as if, ab-initio, it is an open question, i. e. a question that could be answered by making observations, and that one POSSIBLE range of observations will result in a no answer and another possible range of observations will result in a yes answer. But what is the set of observations that results in a no answer to the question Is there anything that cannot be observed. There isn't one. All of the arguments that consciousness can be proven by observation rest on a tacit assumption (dishonestly sneaked in) that observations can answer the question. Nonsense. If the answer were no, you could not verify that the answer is no by observation.
reply
alanhindle3149
I saw a rock on the beach with dents and marks that, in the right light, looked like a human face. Therefore it must have been alive. I saw patterns in a tree's bark that looked like a face, so it must be a troll or ent. I saw a cloud that looked like a turtle, so it must have been a flying turtle. Animals often show characteristics that I can imprint human qualities onto, so they must be just like me. A computer can spit out a statement typed into it by a programmer, so it must be capable of human thought.
It seems to me, and this is only my opinion, that the belief AI is capable of human thought is another example of our human tendency towards animism, and to view patterns and assign them greater meaning. If you are talking to a computer and you cannot tell whether you are talking to a person or not, you are talking to the person who programmed the computer.
reply
I saw a rock on the beach with dents and marks that, in the right light, looked like a human face. Therefore it must have been alive. I saw patterns in a tree's bark that looked like a face, so it must be a troll or ent. I saw a cloud that looked like a turtle, so it must have been a flying turtle. Animals often show characteristics that I can imprint human qualities onto, so they must be just like me. A computer can spit out a statement typed into it by a programmer, so it must be capable of human thought.
It seems to me, and this is only my opinion, that the belief AI is capable of human thought is another example of our human tendency towards animism, and to view patterns and assign them greater meaning. If you are talking to a computer and you cannot tell whether you are talking to a person or not, you are talking to the person who programmed the computer.
reply
tomdouge6618
AI has no lived experiences. They are incapable of drawing on data that they are not given. Yet. They have no understanding of how the real world operates. They do not see, for example, how their generated images may not match a real image. The more sensory details they are to generate means the more their representation is likely to be off. They have no way for correcting the mismatch without input it may not have. And considering how living things can misrepresent reality themselves, that might be a good thing. They are unlikely to have preconceived notions. They can more easily find corrections if the data is broader than the living thing has or is capable of having. Perhaps. They can 'wish' to please the one they are addressing, but it may very well have data the addresser does not have or want. The data will still be there, waiting
reply
AI has no lived experiences. They are incapable of drawing on data that they are not given. Yet. They have no understanding of how the real world operates. They do not see, for example, how their generated images may not match a real image. The more sensory details they are to generate means the more their representation is likely to be off. They have no way for correcting the mismatch without input it may not have. And considering how living things can misrepresent reality themselves, that might be a good thing. They are unlikely to have preconceived notions. They can more easily find corrections if the data is broader than the living thing has or is capable of having. Perhaps. They can 'wish' to please the one they are addressing, but it may very well have data the addresser does not have or want. The data will still be there, waiting
reply
jaspermcjasper3672
Another fallacy is that we can understand what goes on in the human mind by understanding what goes on in the human brain. The two are not the same. Every possible event or state in a brain may map one-to-one both ways with every possible event or state in the mind of the person who's brain it is, but these one-to-one mapping in no way shape or form proves that the two are the same. For every point on the Earth, there is exactly one, and only one, point that is the exact antipode of the first point. Just as this unique two-way one-to-one correspondence doesn't enable us to state that a location on the Earth is the same thing as its antipode, we similarly cannot state that a brain-state or brain-event is not the same thing as the mind-event or mind-state that is occurring when that brain-state or brain-event occurs.
reply
Another fallacy is that we can understand what goes on in the human mind by understanding what goes on in the human brain. The two are not the same. Every possible event or state in a brain may map one-to-one both ways with every possible event or state in the mind of the person who's brain it is, but these one-to-one mapping in no way shape or form proves that the two are the same. For every point on the Earth, there is exactly one, and only one, point that is the exact antipode of the first point. Just as this unique two-way one-to-one correspondence doesn't enable us to state that a location on the Earth is the same thing as its antipode, we similarly cannot state that a brain-state or brain-event is not the same thing as the mind-event or mind-state that is occurring when that brain-state or brain-event occurs.
reply
Corbald
I feel like an obvious oversight of the Chinese Room experiment is that it'd always produce the same output for any given input. I also feel like the potential to 'test' the system is high, and the capacity of the occupant to abandon the manual, learn the language, and communicate outside the room is higher than the thought experiment give credit.
I think, though, that the actual answer has nothing to do with these points, at all. Instead it has to do with 'narrative. ' The 'inference engine' of an LLM is not an intelligent, conscious, aware being. But there's a possibility that any _character_ instantiated by that engine, given sufficient scaffolding in terms of structure, like memory systems or architectures, can arguably be considered to have rudimentary versions of intelligence or consciousness.
reply
I feel like an obvious oversight of the Chinese Room experiment is that it'd always produce the same output for any given input. I also feel like the potential to 'test' the system is high, and the capacity of the occupant to abandon the manual, learn the language, and communicate outside the room is higher than the thought experiment give credit.
I think, though, that the actual answer has nothing to do with these points, at all. Instead it has to do with 'narrative. ' The 'inference engine' of an LLM is not an intelligent, conscious, aware being. But there's a possibility that any _character_ instantiated by that engine, given sufficient scaffolding in terms of structure, like memory systems or architectures, can arguably be considered to have rudimentary versions of intelligence or consciousness.
reply
punkdigerati
Aren't we quite a bit early to be asking if computers have consciousness There currently aren't any of these systems that are being operated in an analogous way to our main example of consciousness, brains, with a stream of input and output and some kind of working memory. Would a brain ran stepwise still display what we call consciousness If it's an emergent phenomenon, the system it emerges from would likely need to at least be somewhat similar operationally to what it's being compared to. We can say that we don't know exactly what's going on under the hood while an AI model is processing a query, but we can certainly say we know what it's doing when it's not processing a query: nothing. This is very different from a human on the other side of a Turing test.
reply
Aren't we quite a bit early to be asking if computers have consciousness There currently aren't any of these systems that are being operated in an analogous way to our main example of consciousness, brains, with a stream of input and output and some kind of working memory. Would a brain ran stepwise still display what we call consciousness If it's an emergent phenomenon, the system it emerges from would likely need to at least be somewhat similar operationally to what it's being compared to. We can say that we don't know exactly what's going on under the hood while an AI model is processing a query, but we can certainly say we know what it's doing when it's not processing a query: nothing. This is very different from a human on the other side of a Turing test.
reply
hughbarton5743
I am absolutey certain that AI, or any foreseeable next generation of computer intelligent cannot achieve. An AI nav system in your car, can give you diirections from Philly up to Fenway Park in Boston. Here's the problem.
Your AI has no idea of
how does a car run
This one happens to run on gasolne.
what's gasoline
Its a flammable liquid that the car burns.
a couple more question:
what is a liquid
what is burning
what is a car.
now, you have not even gotten out of the driveway. An average 5 yesr old, told the family is going to Massachusetts to see a ball game, would have just known about baseball, cars, roads, gas stations, weather delays.
AIs frequently farkup spillchick funkshun.
what is
reply
I am absolutey certain that AI, or any foreseeable next generation of computer intelligent cannot achieve. An AI nav system in your car, can give you diirections from Philly up to Fenway Park in Boston. Here's the problem.
Your AI has no idea of
how does a car run
This one happens to run on gasolne.
what's gasoline
Its a flammable liquid that the car burns.
a couple more question:
what is a liquid
what is burning
what is a car.
now, you have not even gotten out of the driveway. An average 5 yesr old, told the family is going to Massachusetts to see a ball game, would have just known about baseball, cars, roads, gas stations, weather delays.
AIs frequently farkup spillchick funkshun.
what is
reply
walterengler5709
The basic flaw (the bias they mention) is an underlying assumption that there is a definable model of Consciousness, which we can derive from the single limited human model (out mind, our way of thinking. But that ignores how there may be differing models, differing ways of experiencing the sum of the parts, to produce differing forms or methods of creating the subjective experience. Hence you could easily develop and AI that is growing and learning and has reached a new form of consciousness which we would never recognize as it is different, alien in nature. And this means the most likely outcome is Surprise as we have no way of ever seeing it coming. Sleep tight thinking about that.
reply
The basic flaw (the bias they mention) is an underlying assumption that there is a definable model of Consciousness, which we can derive from the single limited human model (out mind, our way of thinking. But that ignores how there may be differing models, differing ways of experiencing the sum of the parts, to produce differing forms or methods of creating the subjective experience. Hence you could easily develop and AI that is growing and learning and has reached a new form of consciousness which we would never recognize as it is different, alien in nature. And this means the most likely outcome is Surprise as we have no way of ever seeing it coming. Sleep tight thinking about that.
reply
Josh-99
Remember: to modern AI, EVERYTHING is just static. It doesn't understand anything, all it does it take inputs (or prompts, if you prefer, convert that input into static it understand (generally a string of numbers) and then match that static to other static that it provides as output. When you train the AI, all you are doing is creating the relationships between incoming static and outgoing static.
This is why AI models start out terrible and get better over time: the more they are used, the stronger the relationships between all of that static become. When you thumbs up or thumbs down an output to your prompt, you are helping to reinforce those relationships -- both positive and negative.
reply
Remember: to modern AI, EVERYTHING is just static. It doesn't understand anything, all it does it take inputs (or prompts, if you prefer, convert that input into static it understand (generally a string of numbers) and then match that static to other static that it provides as output. When you train the AI, all you are doing is creating the relationships between incoming static and outgoing static.
This is why AI models start out terrible and get better over time: the more they are used, the stronger the relationships between all of that static become. When you thumbs up or thumbs down an output to your prompt, you are helping to reinforce those relationships -- both positive and negative.
reply
glenm99
It feels unfair to require human-like (or even animal-like) thinking to conclude that something has consciousness or sentience. Can a thing respond continuously in some manner to its senses, and adapt its behaviour over time in some way that is not necessarily predictable That seems a more general test.
In the Searle experiment, the person locked in the room would be following a pre-defined procedure. Even if there was a mechanism for changing the response when given the same input twice, that input could be given repeatedly until the algorithm for response was figured out. But a sentient being wouldn't respond in a truly predictable way.
reply
It feels unfair to require human-like (or even animal-like) thinking to conclude that something has consciousness or sentience. Can a thing respond continuously in some manner to its senses, and adapt its behaviour over time in some way that is not necessarily predictable That seems a more general test.
In the Searle experiment, the person locked in the room would be following a pre-defined procedure. Even if there was a mechanism for changing the response when given the same input twice, that input could be given repeatedly until the algorithm for response was figured out. But a sentient being wouldn't respond in a truly predictable way.
reply
Scott-sx1bf
I was intrigued by the discussion of bias and it made me wonder whether bias and consciousness can ever be separated. I suspect that bias is an inherent quality of consciousness. If so, what does that imply if/when AI does become conscious Because a conscious AI can learn and reason far more quickly than humans, it would seem that its bias would always be a primary factor in its conclusions. And, it seems logical that those biases would not just be sub-optimal in terms of objectivity but they would also be self-serving to the point of posing a threat to humans dependent on AI in critical situations. Does that therefore become inevitable
reply
I was intrigued by the discussion of bias and it made me wonder whether bias and consciousness can ever be separated. I suspect that bias is an inherent quality of consciousness. If so, what does that imply if/when AI does become conscious Because a conscious AI can learn and reason far more quickly than humans, it would seem that its bias would always be a primary factor in its conclusions. And, it seems logical that those biases would not just be sub-optimal in terms of objectivity but they would also be self-serving to the point of posing a threat to humans dependent on AI in critical situations. Does that therefore become inevitable
reply
Add a review, comment
Other channel videos
