LLMs Ex Nihilo

 WHAT'S IT ALL ABOUT ? SERIOUSLY ? WHEN YOU GET RIGHT DOWN TO IT ?

– Death, Soul Music (Terry Pratchett)

That's the scale of the problem we're facing when dealing with the issue of whether computers can think. Forget the robot uprising crap, we're talking full-on "meaning of life stuff" here. Allow me to explain.

As I've said previously, I've made peace with the idea that we'll probably never get a "truth engine" that can analyse data and form a truly objective, impartial conclusion. We might well get something that does better than we do at formulating conclusions though, which is not to be sniffed at. So instead I'm comfortable with describing AI (mostly but not entirely meaning LLMs) as "thinking engines" instead. That is, they process data and produce conclusions, and that's all I mean by "think". I'm using the word in a very broad, very liberal sense.

I tend to object rather strongly to the notion that LLMs currently go any way beyond this. I don't think they're conscious because I don't think they have any sort of subjective inner awareness. They're not really deterministic but they are still essentially algorithmic robots. They can be described as having a kind of understanding in a useful, productive sense, but not at all in the human-like way. They don't form mental images. They have no sense of the ineffable about them, no emotions, no desires of their own, they are nothing but data processors – and I think there's a very great deal more in a genuine mind than merely analysing raw input data.

The following article from Vox is remarkably thoughtful for a popular piece with a clickbaity headline. It's worth reading in full, but there are two essential points I want to highlight. The first is that minds are not necessarily all alike :

Radically different materials can execute the same basic operation. Biology may have produced the first flying entities. But the reason that birds can soar above the treetops isn’t that they’re made of organic tissue — it’s that their wings perform a set of aerodynamic tasks, such as generating lift and minimizing drag. As airplanes vividly demonstrate, if you put metal and fuel together in just the right way, you can replicate these functions and take to the skies.

To be sure, biological neural networks and artificial ones aren’t identical in design or behaviour. But neither is a cardinal [presumably meaning the bird, not the bishopy sort] and a Boeing 747. Nonetheless, the airplane replicates the avian functions that are necessary for flight. Likewise, computational functionalists wager that computers can instantiate all the neural operations that are relevant to consciousness. So, as long as they recreate a brain’s elaborate algorithms with sufficient precision, they actually can be conscious.

I will gloss over that "be" for the moment, because I take the point that minds might be qualitatively different from each other. Indeed, it would be surprising if they were not, animals having enormously different biology, senses, appendages, and environments with which they have to interact. So I'm prepared to concede that a conscious LLM may have little enough in common with the mind of a bumblebee or a dog or a professor of geology.

But how fundamental can this difference be before we stop calling it a mind at all ? For sure, it could be so different that we have very great difficulty in recognising an alien or robotic intelligence as being conscious. It seems to me, however, that if it doesn't have subjective, inaccessible, inner awareness, if it can't imagine things and those imaginings have no direct correspondence in the real world (just as we can imagine yellow without yellow being present), then we have no business calling it a mind, really. Thinking ? Understanding ? Yes, but only in the (still useful !) sense of data processing. Not at all in the philosophical sense. I see no reason whatever to presume that a network of probabilistic word generation algorithms, however elaborate, would count as truly having a mind any more than would an abacus. 

The article then explores (quite nicely) the notion of computational functionalism, wherein it's the process that's conscious, not the thing itself. I'm sympathetic to this view. Indeed, this could be interpreted as far more subtle than ordinary materialism or physicalism, in which some physical thing actually is conscious. You could even see it as a hybrid step, as close to dualism as a physicalist interpretation would allow*. And in that sense, I don't think it's crazy.

* That is, an electron is a real physical thing. But the mere movement of an electron isn't a thing. You can't say that movement is a physical object, because that's nonsense. So the process of electrons moving around, in a complex EM field, as being conscious is not really the same claim as saying the electrons themselves are actually little bits of minds.

The article's second major point is where I come unstuck :

We can name the physical laws that enable birds to get off the ground. And we have always had reason to believe that inanimate objects could emulate their movement; grains of sand have travelled through the air since time immemorial. By contrast, no one has ever seen a rock experience pain or pleasure, even momentarily (in part, because it’s impossible to directly observe the internal experience of any being or entity other than oneself).

There's the crux. You can't observe consciousness from the outside because it only exists internally, and whether this is because it's a process or an actual substance makes no difference at all. If you see a bunch of neurons firing, do you see yellow ? No, a bunch of neurons firing doesn't look yellow. If you see hormones diffusing throughout the brain, can you honestly say, "yep, I'm witnessing anger ?". No, you cannot. You have no idea what's going on in the conscious mind of the subject. 

The physical and the conscious clearly have some deep, intimate connection, but they are absolutely nothing like each other. There's a world of difference between saying that one gives rise to the other (I'm perfectly comfortable with that) and saying that the two things are literally the same.

A possible escape route is that maybe we can't say a person is angry or horny or introspective from a mere brain scan due to sheer complexity. Maybe we need to know everything in great detail, and we just can't capture the full neurochemical state in enough resolution to make more than a crude guess as to what a person is thinking. Perhaps if we could do so, we would literally be able to read minds.

I claim that this is not the case. I think the Hard Problem is actually the Impossible Problem. I claim that even if we had absolutely all physical information about the system, we would still never know what a person was thinking, not really.

Imagine, if you will, a stupendous spreadsheet listing the configuration of every atom, every subatomic particle in a human body. Could you use this to learn an awful lot about brain states, even a reasonable guess as to what the person was thinking about ? Yes, certainly. But no list of numbers ever actually is yellowness or being horny. The numbers [255, 255, 0] may describe perfectly to a computer how to produce the colour yellow, but it makes no sense to say that that list actually is yellow. They give rise to it, and that is all they do, all they can do.

"What ? That argument is stupid, I'm seeing [255, 0, 0] right now !" said no-one ever. You cannot experience a colour just from a list of numbers : they might give rise to your internal state, but that's not at all the same as saying they actually are the colour itself. That would be insane. Complexity is simply not the issue here at all.

Which brings me to the "meaning of life stuff" I promised earlier. What's been running around in my head is the opposite case : the idea that connections and processes are literally all there is. In that case, you could certainly say that an LLM had a similar consciousness to a human. You wouldn't have to resort to panpsychism, because you could legitimately say that only certainly processes were complex, and not everything in existence undergoes such processes.

But the result would be, I think, extreme nihilism of such a degree as to be actually offensive. I've postulated that meaning arises through knowledge of the connections between different things, that we can be said to understand a thing based on how much we know about how it relates to other things and our ability to predict its behaviour in novel situations. The more moral sense of the word "meaning" may encode something similar : meaningful experiences are those which affect many of our internal links, our own local knowledge of how things relate to each other.

Now suppose that those connections are all there are. That is, knowledge, understanding, and ultimately consciousness itself, arise from connections between items of information and nothing else. That's it. We've hit rock bottom. A mind is a set of relations between physical things and there's nothing else to it. Mystery solved, we can all go home for tea and biscuits.

Or, quite honestly, we may as well all do whatever the fuck we want, or just not bother. I find this idea unspeakably bleak, because if that's all there is, then we're just... electrons and magnetic fields. Atoms bashing about. We have no more claim to moral value than a rock. Without something deeper than the visible connections (some continuing series of connections, possibly going all the way down), we could not rightly say that murder or torture was wrong, and that I simply will not accept.

"But,", you may say, "maybe the moral meaning is something that emerges from the system if it can't be found within it reductively". Ah yes, you might say that. But that concedes the point that there's more to the system than what's visible, doesn't it ?

Fortunately for my own mental health, I think this idea has absolutely no merit in it. The idea that a mind literally is the observable processes, or the physical constituent parts of the nervous system, is to me absolutely and incomprehensibly WRONG. 

Now to be sure many people accept the materialistic world view without falling into moral degeneracy; some at least have probably even fully embraced the "it's all just atoms" perspective without becoming absolute cunts (though I would be a wee bit skeptical as to just how many have really thought this through). And that's fine, I guess, as long as it helps them get through the day. But I cannot escape the feeling that for me personally, this doesn't work. If other people want to live in such a reality, then good for them, I suppose. But for me ? I think I'd rather hit myself in the face with a brick, thanks. After all :

If droids could think, there'd be none of us here.

– Obi-Wan Kenobi, Star Wars