A very interesting, dense article that will require multiple reads.
My only immediate comment is that it talks about qualia being "experienced" without defining what the nature of that experience is. And it kind of assumes that qualia have an ultimate physical manifestation - rather than being merely a rhetorical device.
Also, this site would really benefit from an rss/atom feed.
> And it kind of assumes that qualia have an ultimate
> physical manifestation - rather than being merely a
> rhetorical device.
Denial of Qualia has long been an outlier position in the philosophy of mind, so this is not altogether surprising. (This is saying nothing, of course, about whether it is true or not.)
I am unable to afford the attention at the moment to assess this argument, that's to its credit. They demonstrate that they are aware of the implications of the varying descriptions of consciousness. Most articles I read on this topic turn it to be crude justifications of a naive ideology. This article is a cut above.
I will read it in more detail when I can afford more time.
The existence of qualia does not mean the existence of moral facts.
An llm is in principle implementable with a large lookup table where the prompt is the hash. And it can be implemented in an emulator running an emulator running an emulator of the operating system the llm app runs in. How far are you willing to stretch this substrate idea?
"we cannot be certain that the subjective experience which they [digital minds] may be having is like what we imagine it to be like"
It is even more interesting: we cannot be certain that other people have the subjective experience like we imagine it to be like. It seems to me that the rest of author's arguments implicitly assume that we can be certain about other people's subjective experiences, but if you take this assumption away the arguments can't explain why digital consciousness hypothesis is any different from the hypothesis that the Universe contains at least one person besides me.
I mean I see no point in proving that LLM is not conscious if your methods do not enable us to prove that there are at least two people who is conscious.
I'm so bored with the "are machines conscious" debate. The only thing it has proven is how desperately humans will continue to move the goalposts to the point of ontological collapse in order to preserve their internal sense of primacy.
You don't want to call machines conscious? Fine. Call it quasiconscious, semiconscious, technoconscious, whatever makes you feel alpha human, and move on to our current reality where we confront the existence of thinking machines rather than footnote it into obscurity.
I fully agree with this. I find it strange how quickly the novelty of AI chatbots has passed. This used to be a thing of science fiction, that people dreamed, fantasized about. Now they are everywhere, people rely on them for the most mundane of tasks, and the only debates happening about them are - is their water use efficient enough? Is this CEO likable enough? There seems to be no sense of wonder. Do androids dream of electric sheep? Do humans still dream at all?
> I find it strange how quickly the novelty of AI chatbots has passed.
I had an eye surgery last year. The first day after was magical, I felt like I was on acid because the entire world felt different.
I went for a checkup one week later and told the doctor the vision has gone down since. They checked and it was objectively the same as on the first day. They told me: "People get used to good vision very fast".
Most sci-fi is about genuinely sentient AI, and we are far from that. There is no sign of any intent or internal motivation. If you know anything about machine learning, you know that the idea that LLMs have any form of consciousness is ridiculous. Humans just like to anthropomorphise, that is all.
> There is no sign of any intent or internal motivation.
I don't think that the way we frame the question about consciousness makes it easy to resolve.
But I do want to point out that humans are very diverse. There're many functional adults lacking any internal motivation besides getting food, sleep, and some kind of entertainment. There're millions of people who just act on external stimuli.
So the gap between the "least conscious" human and the "most conscious" machine might be more narrow than we expect.
This is exactly the thought-terminating cliché that I see everywhere, and it's driving me up a wall. We have absolutely no idea what consciousness is, but everybody is 100% sure that LLMs don't "have any form" of it.
Yes, I know about machine learning, but I also know about how natural biological systems learn. It is, just from what we know, very different, yes, but the crucial point is that it, also, has zero identifiable things that you could point at and say: oh, look, that over there is consciousness. So the fact that there is no such thing in an ANN proves nothing.
Mass Effect distinguishes between AI, which is sentient enough to be a person; and VI (Virtual Intelligence) which is more like a conversational user interface. What we have built in real life may qualify as a VI by the Mass Effect standard, but it's a crappy one.
I never thought I'd have so much sympathy for Kate “Forgive me, Mr Data; I am not accustomed to work with non-living devices that… Forgive me again. Your service record says that you are alive.” Pulaski, but here we are.
I like the debate because I've learned a lot of stuff about consciousness, thinking, free will, determinism, agency, intelligence, emergence, problems with reductionism, qualia, etc... Lots of these concepts may be related but it might just be a correlation.
For example, free will and determinism may be orthogonal concepts (compatibilism).
Determinism is interesting. It's mostly a question of 'deterministic in what context'.
Eg the rules of chess are written as if the players have free choice. Similarly, algorithms are often analysed as if the antagonist that provides the data has free choice (so we assume the worst case to be sure).
That's a very different non-determinism from assuming a probability distributions.
And different frames might be accurate depending on context. You might know what chess bot you are playing against and that chess bot might be a deterministic computer program, or might be a computer program that uses radioactive decay to randomise.
And as you suggested, free will is yet another kettle of fish.
I agree that the goalpost is moving all the time. But the whole debate is a red herring. We don't have a good definition for consciousness. I can't really quantify the difference between a bee and a dog.
I would suggest to focus further research on dogs! Why are they so much more inteligent than humans and machines! Dogs are the ultimate stage of the evolution!
They have the real photonic futuristic hardware you are looking for!
Why though? There are only two reasonable reasons I can think of:
1. Scale. There's clearly a scale of consciousness (a jellyfish isn't very conscious but an ape is). Current LLMs are so small (on the scale of brains) that they must be not very conscious at all.
2. Structure. There might be some fundamental structure of computation that is required for consciousness that LLMs don't have yet. For example the ability to form memories (they don't really have that), or some structure of weights that gives consciousness.
My guess is it's definitely 1. and probably also 2.
But I don't think that means thinking LLMs are conscious is completely ridiculous, just a bit premature. I would not be surprised if they are conscious in 50 years.
The real question is how would we ever know that they are. The only reason we know animals are conscious is by direct experience, and we'll never have that for AI.
Anyway this blog post looks like typically worthless philosophy.
Indeed a classic (upvoted!), but it presupposes that there is a theory of everything that can be used to perfectly simulate the universe, and I'd like to remind everybody that we still don't know whether such a thing exists. Nor do we know whether it's possible to simulate "just" neurons and get meaningful results on some higher level. People have been trying at that for decades, so far without success. (nice overview here: https://ccli.substack.com/p/the-biggest-mystery-in-neuroscie...)
Thank you for the link to the post. It was a fascinating read and also entertaining.
After seeing the story of the fruit fly brain simulation I figured stuff like the nematode would be trivial.
The lack of active plasticity in LLMs causes a lot of frustration imo. Having agents make the same mistakes over and over and not be able to correct course without prompt engineering is the biggest symptom.
Obviously a static mind is more helpful from a performance perspective, especially if you consider model deployments into gateware.
Maybe this will be an inherent limitation to LLMs that won’t be solvable without a new approach.
> [...] but it presupposes that there is a theory of everything that can be used to perfectly simulate the universe, [...]
LLMs are nice here: we don't need a theory of everything to simulate them. You can indeed use pebbles on the beach.
Scott Aaronson said that in theory a finite look-up table can win eg a ten minute Turing test (any finite length is fine). Of course, to borrow his argument further, the naive table would be galactic in size. So the real magic is in being able to compress the responses of that table into something human sized / LLM-server sized.
> but it presupposes that there is a theory of everything that can be used to perfectly simulate the universe, and I'd like to remind everybody that we still don't know whether such a thing exists
Is it even possible that a complete simulation of the rules of this universe (let alone the content of it) could exist in this universe?
I am no expert but I did manage to read most of GEB [0] so I feel fully qualified to say that it feels like something Gödel would have opinions about?
[0] the honest nerd's "did sleep in a Holiday Inn last night"
As an analogy: you can have a zip archive that expands to itself. Or even more so: to itself (the simulation of itself in itself) plus something arbitrary (standing in for the rest of the universe).
It is entirely possible (if perhaps unlikely) that the correct theory of everything is a relatively straightforward expansion of the current standard model.
I would argue that for the situation of the comic to occur, it's only necessary to have a theory of everything that's sufficiently good to simulate a universe that's close enough to ours that it's hard to notice the difference.
Our bodies could just be rocks on a beach, but whatever it is about us that is capable of observing the output of our own internal thinking (preventing us from just being p-zombies) is not contained within the beach.
Well, thermodynamics is considered pretty fundamental, and the second law of thermodynamics is pretty much all about information. (But your link already touches that.)
People also interpret the 'unitarity' of quantum mechanics to mean that information is preserved.
I think you're being downvoted because you're announcing it as an absolute truth, of which we do not have evidence.
However I am open to your belief (and not on religious or theistic grounds which I assume many people might think you're talking about) because people have (non-religious) experiences that lend to such possibilities.
yes and no. In my opinion, consciousness is not a process (yes I think Descartes was wrong). The 'thing' that is conscious, whatever it is, is the thing that is capable of experiencing the world. It doesn't need to do anything else to be conscious. Panpsychism is the closest thing to anything that make sense to me (https://en.wikipedia.org/wiki/Panpsychism). This view also makes it possible to separate consciousness from intelligence.
Sounds like you might also be tending towards Illusionism (https://en.wikipedia.org/wiki/Eliminative_materialism#Illusi...) the idea that "Consciousness" isn't a real "thing", or even a real property of a thing, just the name we give to one pattern that emerges from adaptive reasoning systems.
I think that Illusionism and related philosophies will tend to be seen as "correct" once AI systems become more and more commonplace, and people will be forced to dismiss the whole concept of "consciousness" along with mind/body dualism in general as a massive category error that philosophers have been making since antiquity.
When you get down to it, philosophy of consciousness mostly exists because philosophers have assumed that because "mind", "soul", "consciousness" etc are nouns in our language, and that since nouns are "persons places and things" and mind is a noun, mind must be a thing, which is of course not how language works and not sound logic.
I haven't read the article myself yet as it's big and I haven't got time now, but I will as it looks like it might be very good. However, if you read just the very first paragraph you'll see that's exactly what the author thinks:
> Personally, I consider myself a physicalist, but I’m also a panpsychist – because physics doesn’t really seem to deal in hard absolutes, and I find it straightforward to consider that everything is conscious to some greater or lesser degree
Time for a thought experiment! Imagine there are 2 monkeys. One monkey genuinely believes it is a unique and special being. The other monkey is genuinely egoless, believes all monkeys are just a coincidental blob of matter with no fundamental claim to being special and that in some sense are all part of the same being. There is enough food for 1 monkey. If that is all the information we're getting, I would contend that the egotistical monkey is far more likely to be the one to eat the food.
You might easily be you because if you weren't it'd be an evolutionary dead end. Everything with the ability to make choices ahs to believe at some level that it has more of a right to exist than the systems they consume energy more.
Each consciousness is a universe on its own. Matter is just how those universes interact with each other. This is how quantum mechanics work in my idea, each observer affects the matter and quantum effects take most probable outcomes because of that.
"When I saw this animation I was immediately inspired to write an impressionistic tweet about it"
Maybe we should set the threshold for consciousness a little bit higher
A very interesting, dense article that will require multiple reads.
My only immediate comment is that it talks about qualia being "experienced" without defining what the nature of that experience is. And it kind of assumes that qualia have an ultimate physical manifestation - rather than being merely a rhetorical device.
Also, this site would really benefit from an rss/atom feed.
myth of the given strikes again
I am unable to afford the attention at the moment to assess this argument, that's to its credit. They demonstrate that they are aware of the implications of the varying descriptions of consciousness. Most articles I read on this topic turn it to be crude justifications of a naive ideology. This article is a cut above.
I will read it in more detail when I can afford more time.
The existence of qualia does not mean the existence of moral facts.
An llm is in principle implementable with a large lookup table where the prompt is the hash. And it can be implemented in an emulator running an emulator running an emulator of the operating system the llm app runs in. How far are you willing to stretch this substrate idea?
"we cannot be certain that the subjective experience which they [digital minds] may be having is like what we imagine it to be like"
It is even more interesting: we cannot be certain that other people have the subjective experience like we imagine it to be like. It seems to me that the rest of author's arguments implicitly assume that we can be certain about other people's subjective experiences, but if you take this assumption away the arguments can't explain why digital consciousness hypothesis is any different from the hypothesis that the Universe contains at least one person besides me.
I mean I see no point in proving that LLM is not conscious if your methods do not enable us to prove that there are at least two people who is conscious.
I'm so bored with the "are machines conscious" debate. The only thing it has proven is how desperately humans will continue to move the goalposts to the point of ontological collapse in order to preserve their internal sense of primacy.
You don't want to call machines conscious? Fine. Call it quasiconscious, semiconscious, technoconscious, whatever makes you feel alpha human, and move on to our current reality where we confront the existence of thinking machines rather than footnote it into obscurity.
I fully agree with this. I find it strange how quickly the novelty of AI chatbots has passed. This used to be a thing of science fiction, that people dreamed, fantasized about. Now they are everywhere, people rely on them for the most mundane of tasks, and the only debates happening about them are - is their water use efficient enough? Is this CEO likable enough? There seems to be no sense of wonder. Do androids dream of electric sheep? Do humans still dream at all?
> I find it strange how quickly the novelty of AI chatbots has passed.
I had an eye surgery last year. The first day after was magical, I felt like I was on acid because the entire world felt different.
I went for a checkup one week later and told the doctor the vision has gone down since. They checked and it was objectively the same as on the first day. They told me: "People get used to good vision very fast".
I think it's just human nature.
Most sci-fi is about genuinely sentient AI, and we are far from that. There is no sign of any intent or internal motivation. If you know anything about machine learning, you know that the idea that LLMs have any form of consciousness is ridiculous. Humans just like to anthropomorphise, that is all.
> There is no sign of any intent or internal motivation.
I don't think that the way we frame the question about consciousness makes it easy to resolve.
But I do want to point out that humans are very diverse. There're many functional adults lacking any internal motivation besides getting food, sleep, and some kind of entertainment. There're millions of people who just act on external stimuli.
So the gap between the "least conscious" human and the "most conscious" machine might be more narrow than we expect.
Yes, our current systems are likely still to primitive to be properly conscious. Probably.
But I wouldn't rule out that ChatGPT 10 or Claude Fable 15 might get conscious.
You are committing a type error.
Consciousness does not necessarily arise from computation, just as simulating snowfall on the computer does not make it cold
This is exactly the thought-terminating cliché that I see everywhere, and it's driving me up a wall. We have absolutely no idea what consciousness is, but everybody is 100% sure that LLMs don't "have any form" of it.
Yes, I know about machine learning, but I also know about how natural biological systems learn. It is, just from what we know, very different, yes, but the crucial point is that it, also, has zero identifiable things that you could point at and say: oh, look, that over there is consciousness. So the fact that there is no such thing in an ANN proves nothing.
If it drives you up a wall maybe you can look intoning there are any arguments for the positing driving you up there.
Mass Effect distinguishes between AI, which is sentient enough to be a person; and VI (Virtual Intelligence) which is more like a conversational user interface. What we have built in real life may qualify as a VI by the Mass Effect standard, but it's a crappy one.
I never thought I'd have so much sympathy for Kate “Forgive me, Mr Data; I am not accustomed to work with non-living devices that… Forgive me again. Your service record says that you are alive.” Pulaski, but here we are.
I like the debate because I've learned a lot of stuff about consciousness, thinking, free will, determinism, agency, intelligence, emergence, problems with reductionism, qualia, etc... Lots of these concepts may be related but it might just be a correlation.
For example, free will and determinism may be orthogonal concepts (compatibilism).
Determinism is interesting. It's mostly a question of 'deterministic in what context'.
Eg the rules of chess are written as if the players have free choice. Similarly, algorithms are often analysed as if the antagonist that provides the data has free choice (so we assume the worst case to be sure).
That's a very different non-determinism from assuming a probability distributions.
And different frames might be accurate depending on context. You might know what chess bot you are playing against and that chess bot might be a deterministic computer program, or might be a computer program that uses radioactive decay to randomise.
And as you suggested, free will is yet another kettle of fish.
How about I call it not conscious at all, not even a little bit?
I agree that the goalpost is moving all the time. But the whole debate is a red herring. We don't have a good definition for consciousness. I can't really quantify the difference between a bee and a dog.
I would suggest to focus further research on dogs! Why are they so much more inteligent than humans and machines! Dogs are the ultimate stage of the evolution!
They have the real photonic futuristic hardware you are looking for!
Thinking LLMs are conscious is as ridiculous as thinking a thermostat wants the room to be warm
Why though? There are only two reasonable reasons I can think of:
1. Scale. There's clearly a scale of consciousness (a jellyfish isn't very conscious but an ape is). Current LLMs are so small (on the scale of brains) that they must be not very conscious at all.
2. Structure. There might be some fundamental structure of computation that is required for consciousness that LLMs don't have yet. For example the ability to form memories (they don't really have that), or some structure of weights that gives consciousness.
My guess is it's definitely 1. and probably also 2.
But I don't think that means thinking LLMs are conscious is completely ridiculous, just a bit premature. I would not be surprised if they are conscious in 50 years.
The real question is how would we ever know that they are. The only reason we know animals are conscious is by direct experience, and we'll never have that for AI.
Anyway this blog post looks like typically worthless philosophy.
No matter how large the llm hardware is it won’t be made of brain meat. The stuff we actually know is conscious.
Or that a slug dislikes salt lol.
Reminds me of this classic: https://xkcd.com/505/
Replace universe simulation with a LLM an it’s hard to consider a bunch of rocks shuffling on a beach as conscious.
Or alternatively, our own minds are simply (squishy, organic) rocks on a beach, which is quite humbling if you think about it long enough.
Indeed a classic (upvoted!), but it presupposes that there is a theory of everything that can be used to perfectly simulate the universe, and I'd like to remind everybody that we still don't know whether such a thing exists. Nor do we know whether it's possible to simulate "just" neurons and get meaningful results on some higher level. People have been trying at that for decades, so far without success. (nice overview here: https://ccli.substack.com/p/the-biggest-mystery-in-neuroscie...)
Thank you for the link to the post. It was a fascinating read and also entertaining.
After seeing the story of the fruit fly brain simulation I figured stuff like the nematode would be trivial.
The lack of active plasticity in LLMs causes a lot of frustration imo. Having agents make the same mistakes over and over and not be able to correct course without prompt engineering is the biggest symptom.
Obviously a static mind is more helpful from a performance perspective, especially if you consider model deployments into gateware.
Maybe this will be an inherent limitation to LLMs that won’t be solvable without a new approach.
> [...] but it presupposes that there is a theory of everything that can be used to perfectly simulate the universe, [...]
LLMs are nice here: we don't need a theory of everything to simulate them. You can indeed use pebbles on the beach.
Scott Aaronson said that in theory a finite look-up table can win eg a ten minute Turing test (any finite length is fine). Of course, to borrow his argument further, the naive table would be galactic in size. So the real magic is in being able to compress the responses of that table into something human sized / LLM-server sized.
> but it presupposes that there is a theory of everything that can be used to perfectly simulate the universe, and I'd like to remind everybody that we still don't know whether such a thing exists
Is it even possible that a complete simulation of the rules of this universe (let alone the content of it) could exist in this universe?
I am no expert but I did manage to read most of GEB [0] so I feel fully qualified to say that it feels like something Gödel would have opinions about?
[0] the honest nerd's "did sleep in a Holiday Inn last night"
> Is it even possible that a complete simulation of the rules of this universe (let alone the content of it) could exist in this universe?
Yes, it is possible! See how a Quine is constructed.
https://en.wikipedia.org/wiki/Quine_(computing)
As an analogy: you can have a zip archive that expands to itself. Or even more so: to itself (the simulation of itself in itself) plus something arbitrary (standing in for the rest of the universe).
There's also information in the compiler and the zip binary, without that I'm pretty sure it's not possible.
It is entirely possible (if perhaps unlikely) that the correct theory of everything is a relatively straightforward expansion of the current standard model.
I would argue that for the situation of the comic to occur, it's only necessary to have a theory of everything that's sufficiently good to simulate a universe that's close enough to ours that it's hard to notice the difference.
Our bodies could just be rocks on a beach, but whatever it is about us that is capable of observing the output of our own internal thinking (preventing us from just being p-zombies) is not contained within the beach.
The "whatever" part, in strict terms, would be information. Jury is still out whether that is a fundamental component of the universe or not: https://www.pbs.org/wgbh/nova/article/is-information-fundame...
Well, thermodynamics is considered pretty fundamental, and the second law of thermodynamics is pretty much all about information. (But your link already touches that.)
People also interpret the 'unitarity' of quantum mechanics to mean that information is preserved.
This makes no sense at all as soon as you start trying to define where the internal/external line actually is.
I think you're being downvoted because you're announcing it as an absolute truth, of which we do not have evidence.
However I am open to your belief (and not on religious or theistic grounds which I assume many people might think you're talking about) because people have (non-religious) experiences that lend to such possibilities.
Are we conscious at hardware/cell level?
yes and no. In my opinion, consciousness is not a process (yes I think Descartes was wrong). The 'thing' that is conscious, whatever it is, is the thing that is capable of experiencing the world. It doesn't need to do anything else to be conscious. Panpsychism is the closest thing to anything that make sense to me (https://en.wikipedia.org/wiki/Panpsychism). This view also makes it possible to separate consciousness from intelligence.
Sounds like you might also be tending towards Illusionism (https://en.wikipedia.org/wiki/Eliminative_materialism#Illusi...) the idea that "Consciousness" isn't a real "thing", or even a real property of a thing, just the name we give to one pattern that emerges from adaptive reasoning systems.
I think that Illusionism and related philosophies will tend to be seen as "correct" once AI systems become more and more commonplace, and people will be forced to dismiss the whole concept of "consciousness" along with mind/body dualism in general as a massive category error that philosophers have been making since antiquity.
When you get down to it, philosophy of consciousness mostly exists because philosophers have assumed that because "mind", "soul", "consciousness" etc are nouns in our language, and that since nouns are "persons places and things" and mind is a noun, mind must be a thing, which is of course not how language works and not sound logic.
My cat is more conscious than whatever latest frontier model is out there, and it runs on a little bit kibble and water.
And if digital computers are conscious at the hardware level, the universe is conscious.
I haven't read the article myself yet as it's big and I haven't got time now, but I will as it looks like it might be very good. However, if you read just the very first paragraph you'll see that's exactly what the author thinks:
> Personally, I consider myself a physicalist, but I’m also a panpsychist – because physics doesn’t really seem to deal in hard absolutes, and I find it straightforward to consider that everything is conscious to some greater or lesser degree
Consciousness could be an emergant property too, of matter arranged in a certain way. Why am I me though? Drives me nuts!
Time for a thought experiment! Imagine there are 2 monkeys. One monkey genuinely believes it is a unique and special being. The other monkey is genuinely egoless, believes all monkeys are just a coincidental blob of matter with no fundamental claim to being special and that in some sense are all part of the same being. There is enough food for 1 monkey. If that is all the information we're getting, I would contend that the egotistical monkey is far more likely to be the one to eat the food.
You might easily be you because if you weren't it'd be an evolutionary dead end. Everything with the ability to make choices ahs to believe at some level that it has more of a right to exist than the systems they consume energy more.
I mean, we are as much a part of the universe as everything else. We're just dead star matter arranged in specific ways.
A part of the universe is objectively conscious.
I have yet another take on that.
Each consciousness is a universe on its own. Matter is just how those universes interact with each other. This is how quantum mechanics work in my idea, each observer affects the matter and quantum effects take most probable outcomes because of that.
And so are electrons.
That’s an interesting thought. I’ll try not to type too hard on my keyboard then
my keyboard enjoys a bit of rough play
The latter is easier to accept.