It's time for another amateur epistemology post, because that's just the kind of thing I like to do in my off-hours.
Previously I've looked at analytical and critical thinking, claiming that the first is about examining consequences (if this is true, what follows ?) while the second is about considering whether something actually is true in the first place. Then I moved on to the more interesting issue of wisdom. I suggest that this is the combination of critical and analytical thinking but applied on multiple scales, considering the whole aspect of a problem, from its most minute details to its wider context and implications.
Along the way, the issue of understanding has cropped up many times. This has been especially contentious in the age of LLMs, which appear to do an increasingly good job of mimicking the human capacity for cognition. But are they really thinking ? Or is it all just a clever imitation, a sort of advanced party trick, but not really anything like how human thinking works ?
As you can see in the previous post and its numerous links, this is one that's been going on for some time, and nobody seems to have a clear idea of what we really mean by "understanding". My own rough idea is that it's the knowledge of the connections between different things, or per that particular post, the ability to construct new connections. If you understand something, you can predict how it behaves in novel situations.
Following another mammoth discussion, I think I can now go further. I think I have a definition which is reasonably complete, sufficiently limited as to be meaningful, and might help answer why we can't just keep learning stuff indefinitely, independently of our memory constraints. I'm also going to claim that yes, chatbots absolutely have a form of understanding which is remarkably similar to, or at the very least closely analogous with, our own.
Rather than mere knowledge of connections, I now think that understanding is a form of judgement. It's the ability to judge :
- When to apply existing methods to solve a problem
- When to acquire new techniques, ideas, and/or data (both from combining existing data in a novel way and by getting brand new data)
- When to accept that an issue is unsolvable
- An awareness of possible missing factors, and judging the possible effects of these.
Hence, understaaaanding. But I imagine you got that already.
How does this help ? Well, it allows an interpretation of how humans proceed with problem solving and therefore we can relate this to what chatbots are doing.
Suppose I’m asked to calculate the distance travelled by a ball chucked at 10 m/s at 45 degrees. No problem ! I can use the classic suvat equations. I’m aware of their limitations and can add air resistance and such if I really want to, but I’ll also assume this is good enough for a first guess. I can refine the answer later, if asked. I'll assume that, by the nature of the question, a simple parabolic estimate is all that's needed.
We can see here that being able to apply the equations unmodified in no way whatever implies I don’t understand them. Nobody would think that. Understanding doesn’t always require forming new connections; mere knowledge of them is sometimes enough. For simple stuff like this, I can do it off the bat almost without thinking.
Then I’m asked to calculate the trajectory of a ball which is thrown at 10 km/s. Now we’re into very different territory. Air resistance might destroy the ball completely, or if not, we’re into the realm of orbital mechanics where good old suvat simply doesn’t apply. I can probably do it, but it’s going to take some mental digging around to work things out. I’ll definitely be making a lot of notes to re-form some long-disused mental pathways. I need to re-acquire unused knowledge and skills, though I'm still aware that I can't just use equations of constant acceleration any more.
Finally, suppose I’m asked to calculate whether any fragments of a projectile ejected from the Solar System at 100,000 km/s will ever reach Alpha Centauri. Now things get phenomenally more complicated. We’ve got to deal with stellar motions and their measurement errors, ablation by relativistic dust gains, thermal effects, directional uncertainty… I understand many of the issues involved, but only well enough to accept that this is a problem beyond my abilities.
Do chatbots demonstrate these sorts of judgements ? Unquestionably yes, at least the modern ones. But first, note I say "chatbot" specifically, not "LLMs". I won't go into the architectural details (but see this), but the essential point is that the chatbot interface is where the user enters text and other files, has a conversation etc., while the LLM is the core of the underlying thinking engine that issues the responses.
This matters, because in that last mega-thread, one particularly controversial point was that the LLM itself isn't updated by conversations. Yes, conversations are used for training new LLMs, but this is much more like writing a whole new book than revising an ongoing draft. A human brain does tend to change in response to new information, an LLM does not. But the chatbot conversation questionably does. Tell it brand new information, sufficiently related to its existing knowledge, and it can absolutely apply this correctly and sensibly. True, it'll forget it all in the next conversation, but that's not the point. The point is that it demonstrates this at all, not whether it's truly human-like.
The main issue here seemed to be that true understanding should involve a change in the system, showing a progression from ignorance to ability to actually apply its new-found comprehension. But I don't believe this is really the case. Human working memory is incredibly brief, and it happens constantly that we might learn something and forget it almost immediately. More fundamentally, we may simply not care. I understand flat Earth ideas perfectly well, I just don't give a rat's arse about them. In fact, I actively try to avoid having them taking up any more mental bandwidth than is absolutely unavoidable. If they affect me at all, it's in an awareness that there are far too many stupid people out there, not in altering my view of the cosmos. Understanding is not the same as acceptance or belief*.
* Chatbots can certainly fake belief just as they can fake emotions. The difference here is that emotion is self-evidently something more than being able to rearrange words, whereas to my mind the definition of "understanding" requires no such missing element.
Chatbots, then, definitely do demonstrate a very human-like capability of understanding, but this is only analogous with human working memory, and doesn't indicate anything like human cognition more widely. They don't really believe or care about anything. They don't truly learn, in that their long-term memory doesn't change*. But defining understanding to includes these facets of human thinking would, I think, be incorporating more than the word can be reasonably expected to bear, potentially making it so broad as to be useless.
* While you can give a chatbot a stack of documents it always reads, which can be updated (and even self-updated by the bot itself), this consumes its context window. This renders them incapable of the kind of learning of which humans are capable, but even ignoring this restriction, the LLM itself is never updated.
In my view, then, there is simply nothing left. Chatbots understand things, but this in no way implies they think like a human. Or perhaps, in some narrow, specific ways, they do think like people, while in others they remain profoundly different. They can judge when existing data exists which already answers a user's question, or when they might need to search for novel connections, or even when to simply accept that the answer cannot be obtained. This barely scratches the surface of cognition more generally, and it's highly functionalist, but I cannot see what more would be necessary to claim understanding specifically.
Two final points. First, a long-term concern for me has been how "knowledge of connections" ties to incomprehensible yet irreducible facts. In particular, why do I encounter limitations ? How come I could get so far in higher mathematics, but no further ?
This new framework helps, I think. It's because I can't judge how new information – new symbols and new processes – relate to my existing mental network. It might be in part because of hardware limitations, just as I've previously speculated (e.g. not being able to hold more than a certain number of connections in place at any one time, or the need to adjust more connections at once than I can sustain). It might also be to do with how I learned things initially : the existing architecture of my own neural network may simply make fitting in new pieces highly demanding. The key difference is that while I might indeed be able to learn an indefinite number of new facts, I can't guarantee that all of them will make sense to me.
In that mega-thread, I raised an analogy of a vast, static database containing the answer to every single question conceivable. Would it understand things ? The answer comes by comparing a nearly-as-vast but dynamic database, one which doesn't hold as much directly, but has the ability to work out the answer to every question rather than regurgitating every single answer. That's where understanding lies, I think : the ability to work things out.
And lastly, how far can chatbot understanding be pushed ? For that I refer to this discussion I had with ChatGPT. After all, merely having understanding does not directly imply "makes a useful contribution to knowledge", any more than we'd expect Wayne Rooney to come up with some great philosophical insight. Which is fair, but I think this contributes a lot to the routine goalpost-shifting standards people apply to LLMs ("Hah, it can't even do simple algebra" one day becomes "Hah, it can't do differential calculus" the next... certain people really do lack rather a lot of that awareness-aspect of understanding).
But if LLMs are trained more on the process of working things out, rather than pure word-association, then their user-facing chatbots can potentially get very far indeed. What might, I suggest, make the issue of whether they really understand things really unavoidable would be a chatbot that goes beyond combing things in a novel and correct way. Just as humans uniquely make tools to make other tools, a bot with next-level understanding would be able to come up with a new concept and remain able to apply this – something which didn't exist until it thought of it – (mostly) correctly in novel situations. I mean something like liberalism, capitalism, relativity, or evolution : the pieces of the puzzles already existed, but the whole became greater than the sum of its parts.
If a chatbot can come up with something like that, surely then even the most evangelically anti-chatbot ought to concede defeat. It would have made a genuine contribution to knowledge. It doesn't have to pluck forth new insights fully formed from the head of Zeus; it can absolutely be allowed to use existing data. But it would have to demonstrate that the new product was something genuinely, qualitatively new. If it could do that (and most of us humans can't !), then surely the goalposts would now be well and truly against a wall.
No comments:
Post a Comment
Due to a small but consistent influx of spam, comments will now be checked before publishing. Only egregious spam/illegal/racist crap will be disapproved, everything else will be published.