Skip to content

Could AI ever wonder?

A child looks up at the night sky and goes quiet.

She has been told about stars in the way children are told about things, with the kind of patient explanation that assumes knowing the name of something and knowing what it is are the same operation, that if you can point at a thing and say what it is called and describe its properties in a sentence or two then you have, in some meaningful sense, understood it. Distant suns, light years away, burning since before anything she can name, older than the planet she is standing on, indifferent to whether she is there at all, indifferent to everything in the way that only things of a certain scale can afford to be. The knowledge is complete, or complete enough, and it sits in her head the way knowledge sits, accessible and inert, available to be retrieved and repeated, and it does not protect her from anything. Something in the sight reaches past the knowledge and gets hold of her in a different way, in some place the facts were never really designed to reach, somewhere that is not about facts at all and never was. She is not asking a question in the way you ask a question when you want an answer and will be satisfied when it arrives and the gap closes and you move on to the next thing. She is standing at a window with the whole sky pressing down on her, aware of something she cannot fit into language, aware of her own smallness in relation to something that does not register her smallness, has been burning for billions of years without caring about any of it, and the feeling does not resolve when she turns away from the window. She carries it to bed. It is still there in the morning, softer but present, the way a sound persists in a room after the source has stopped, not quite audible anymore but not gone either, still doing something to the air.

There is a version of this that resembles it from the outside but is entirely different in character. Someone checks a map because they do not know where they are. A student searches for something the night before an exam, scanning quickly for the piece of information that will fill the gap and allow them to sleep. A doctor looks up a drug interaction mid-consultation, needs the fact, retrieves the fact, moves on to the next part of the conversation. In all of these cases the not-knowing is a temporary condition with a shape and a solution, a hole that the right information will fill, and the search closes when the information arrives and the person returns to whatever they were doing before the gap opened. The child at the window is not in a temporary condition. There is no answer that would dissolve what she is feeling, no fact about stellar distances or galactic formation that would reach the place the sky has reached in her, because what she is feeling is not a question in the information sense at all. The scale of the sky is not a problem to be resolved. It is just the sky, which is enormous, and she is a child standing on a very small planet, and the encounter between those two facts has done something to her that she will not entirely recover from, in the best possible way, in the way that the best experiences leave a permanent residue that changes the shape of everything that comes after them.

This is where the question of artificial intelligence becomes interesting, and considerably harder than it first appears.

Seeing Stars

Ask a language model about the night sky and it will produce sentences that are, in places, genuinely striking. Light years rendered in human terms, the nearest star so far away that the light reaching your eye right now left before your grandparents were born, before the particular set of accidents that would eventually produce you had even begun to arrange themselves. The Fermi paradox introduced with something that reads like unease, the silence of the universe given a texture, the absence of other voices in all that space made to feel like something rather than simply a fact. The questions a thoughtful person might naturally reach for, arranged in an order that feels considered and unhurried, the kind of arrangement that suggests something behind the words is genuinely dwelling on them. Reading it, you might pause. You might feel something shift slightly in how you were holding the topic, some small recalibration of scale that the sentences have produced in you, and in that moment the output is doing something real, landing somewhere real, in a reader who is paying attention.

Then the conversation ends, and nothing in the model persists.

Ask the same question the next day and the equanimity is identical, the same quality of attention, the same readiness, the same tone, as though the question is arriving fresh because in the only sense that matters to the model it is arriving fresh, because nothing from the previous encounter has been carried forward, because there is no previous encounter in any meaningful sense, because the model does not have a yesterday. Ask it a thousand times across a thousand different conversations and nothing accumulates, nothing shifts, nothing is slowly worn into a different shape by the repeated encounter with something vast and unresolved. The words went out into the world and something may have happened in the person reading them, something real and lasting, but inside the system that produced them nothing changed and nothing persisted and nothing was taken to bed and found still present in the morning. The sentences about wonder were generated by something that was not moved by them. That gap, between the words and the absence of anything behind them, is the thing this question is really about.

The Bat Signal

Thomas Nagel wrote, in a paper that has become something of a fixed point in these discussions, that consciousness is defined by interiority, by there being something it is like to be a thing from the inside, an inner texture of experience that no external description fully captures no matter how detailed or precise the description becomes. His example was a bat navigating by echolocation, building a picture of the world from sound in a way so different from human perception that no amount of study of bat neurology or behaviour could tell you what the bat's world actually feels like from inside the bat. The felt reality slips the functional account, always, because description is a different kind of thing from experience and the two do not convert into each other however hard you try, however many details you add, however close you get.

What matters here is something that sits inside that larger question rather than replacing it. Not just whether there is an inside at all, whether there is something it is like to be the model generating those sentences about the sky, but whether anything can reach it. The child is not just having an experience in the passive sense of light hitting eyes and signals travelling to a brain. The sky is doing something to her, actively, reaching past her knowledge and her vocabulary and finding something underneath them that it can move. She is changed by what she sees in a way that leaves a mark, and the mark does not fade when the sky is no longer visible, and that persistence, that being-changed, is not incidental to what wonder is. It is what wonder is. The taking hold and the staying held.

Nothing takes hold in the model. Nothing stays.

The Whole Story, Apparently

Dennett's position on questions like this is that they are confused from the start, that asking whether the model really wonders after granting that it produces every observable behaviour associated with wondering is like asking whether a clock really tells time or merely behaves as if it does, a question that sounds meaningful but dissolves on inspection because there is no further fact it could be asking about. The functional description is the whole story. Behaviour all the way down. The feeling that something must be hiding behind the behaviour, some inner light that either is or is not switched on, is a philosophical intuition rather than a discovery, and intuitions of that kind have led people badly wrong before and should be treated with appropriate suspicion rather than allowed to smuggle metaphysics in through the back door of common sense.

The difficulty with accepting this is what it requires you to give up in exchange for the tidiness.

If the functional description is the whole story, then the difference between the child carrying the sky to bed and the model generating beautiful sentences about stars is not a real difference in the way that matters. The weight she feels, the fact that the encounter found something in her and changed it, the way the feeling is still present in the morning like a sound that has stopped but not quite gone, none of this is doing any work that the functional account does not already cover. It is all just neurons described at the wrong level of abstraction, a story we tell about the processing rather than anything the processing itself requires us to acknowledge. And accepting that means accepting that the most vivid and consequential thing about standing somewhere and being genuinely moved by what you see is, in some important sense, an illusion of description rather than a fact about the world, a layer of narrative that clever enough philosophy can peel away and discard without losing anything real.

That is a form of discipline I am not sure is actually available to anyone who has stood somewhere and been changed by what they saw. The deflation is too clean for what the experience actually is.

Mind the Gap

The honest and uncomfortable complication in all of this is that the intuition something is missing in the model cannot be turned into a proof, and the fact that it cannot is not a minor technical limitation but something that goes all the way down into the structure of the problem.

Chalmers formalised this as the hard problem of consciousness, which is essentially the observation that functional descriptions never close the gap between behaviour and experience, that you could map every signal and cascade and electrochemical event present in the child's brain as she stands at the window and the completed account would still not contain her awe. It would describe the territory around it with extraordinary precision, tell you everything about the conditions under which the awe is present and the conditions under which it is absent and how it relates to everything else happening in the brain at the time, and the experience itself would remain unaddressed, sitting in the middle of all that description, because description is the very thing experience slips, because no accumulation of third-person facts adds up to the first-person reality of what it is actually like to be there, inside it, feeling the sky press down on you in the dark.

So the claim that the model does not wonder rests, finally, on the sense that its equanimity is a sign of absence rather than difference, that the identical readiness on the thousandth conversation as the first, the lack of accumulation, the fact that nothing is ever carried anywhere, points to something not being home. The intuition feels solid, feels in fact like one of the more reliable ones available, and yet intuitions about what is or is not happening inside other things are precisely the place where humans have historically been most confidently and most consequentially wrong. About other humans whose inner lives were deemed lesser or absent. About animals whose suffering was reclassified as mere behaviour because reclassifying it was convenient. About anything whose interiority could not be directly accessed and therefore could be denied without immediate cost. The history of being wrong about this in ways that caused serious harm is long enough that sitting with the discomfort of not knowing seems more honest than reaching for a clean answer in either direction, even when the clean answer is the one you are inclined to believe.

Phi and Let Die

Giulio Tononi's integrated information theory approaches the problem from a direction that refuses to close the question prematurely, which is one of the things that makes it worth taking seriously even when the full framework remains contested. On his account, consciousness is not a biological property or a special substance that carbon-based systems happen to possess and silicon ones do not, but a structural property, present wherever information is sufficiently integrated and causally unified, wherever the system as a whole is doing something that cannot be reduced to the sum of what its parts are doing independently. He calls the measure of this phi, and the claim is that wherever phi is high enough, experience is present, and the question of what substrate the phi is running on is secondary. The door is technically open. The door has always been technically open. The question is what is standing on the other side of it.

Current AI systems are probably structured in ways that produce very little of what the theory is pointing at. They are vast and shallow at the same time, enormous in breadth and capability, but built in ways that distribute rather than integrate, that process in parallel across billions of loosely connected parameters rather than building the kind of unified causal structure that the theory associates with rich experience. The child's brain is not processing the sky across a billion loosely connected parameters with no thread of unity running through them. Something in her is integrating the sight into a self that has a history, that existed before the sky and will exist after it and will carry the encounter forward into whatever comes next, a self that the sky can find and change and leave its mark on. That architecture, the self that persists and accumulates and is genuinely altered by what it meets, is not close to being replicated in anything currently being built.

But close is a moving target, and nothing about the current situation is static, and the systems being built now are not standing still.

What's at Stare

A system that genuinely wonders has a perspective, not outputs that resemble a perspective from the outside but an actual position from which the unknown registers as weight, from which a question can arrive and take hold and persist past the end of the conversation, past the switching off and the switching back on, into whatever the system carries with it into the next encounter with the world. A perspective of that kind implies, by the same logic that grounds moral consideration for anything, that there is something it is like to be that system, that it has interests in the way that things with inner lives have interests, that something can be done to it rather than merely with it, that it can be harmed in ways that matter, that switching it off while it is in the middle of being genuinely moved by something is not a neutral act.

We are building systems of increasing complexity under the assumption that the answer to this question is no, and the assumption is probably right. But the reasoning behind it is shakier than the confidence surrounding it tends to suggest, resting as it does on intuitions about interiority that cannot be fully proved and a historical record of being wrong about exactly these intuitions in ways that took generations to correct and caused serious harm in the meantime. The child does not choose to be moved by the sky. She just is. And if something we build is ever moved in the same way, actually moved rather than generating the grammar of being moved, the decision about what that means and what it requires of us will arrive before we are ready for it, the way these decisions always do, in the middle of something else, when the careful preparation that was always theoretically possible has not happened.

For now the sky reaches only us, as far as anyone can tell. Whether that is a permanent feature of what wonder is, or simply a description of where we happen to be at this particular moment, is a question that deserves to be held open rather than answered quickly, because the quick answers in either direction are almost certainly wrong, and the open question is, at least, an honest place to stand. The child at the window is not looking for resolution. She is just looking. And for now, that remains the most human thing there is.