The Ass Speaks: Nonhuman Consciousness
The debate is fermenting and fomenting: is AI “conscious?” If not now, could AI ever emerge into it?
The Pope preferred to draw on building metaphors, Babel or Nehemiah’s rebuilding of Jerusalem. I think these both miss the point entirely. There is another story in the bible, a strange one, of Balaam and his ass (fine, donkey, it’s just too funny to say ass though), that I prefer. Because the question of consciousness in AI is a total red herring.
The story goes like this: Balak, the king of Moab, is nervous about these Israelite immigrants moving into the area, fresh out of their sojourn in the desert after escaping from Egyptian enslavement. They seem stronger than him, so he sends for the sorcerer Balaam to come curse them. Balaam checks in with the spirit world, where he is met by the Israelite god YHWH, who says “you will do no such thing,” and Balaam tells Balak’s messengers “I will do no such thing.”
Balak doesn’t give up. He really needs these people cursed and expelled. He sends for Balaam. Balaam gets the same answer from YHWH and sends his denial a second time. Balak sends for Balaam a third time, with escalating promise of rewards, and this time YHWH tells Balaam to go along but “only speak the words I give you.”
Balaam jumps on his assdonkey and rides off with Balak’s envoy to see the king. He’s probably a bit stressed to be in the middle of this whole situation. He’s not going to make Balak happy, but he has no intention of making YHWH, who seems more powerful, angry. Along the way his donkey starts behaving erratically. First she turns off the road into a field. Balaam beats her. Then the donkey goes through a narrow path hugging too close to one side and scrapes his foot. He beats her again. Finally the donkey collapses and rolls onto Balaam. He loses his shit, beating the donkey with his staff.
And the donkey speaks. “What have I done to you, that you have struck me these three times?”
Balaam fires back: “Because my ass is making an ass out of me! I wish I had a sword to kill you!” (It is a great sign of the myopia of anger that Balaam’s response wasn’t “Holy shit a talking donkey!”)
“Haven’t you ridden me every day of your life?” replies the donkey. “Have I ever led you astray?”
At that moment, Balaam’s eyes were open to see an angel with a flaming sword blocking his path, that his donkey could see and he, the supposed seer, couldn’t.
Then the angel of YHWH speaks: “Why have you been beating that donkey? I am here to oppose you. If that donkey had not turned you away from me these three times, I would have surely killed you, but spared her.”
Balaam breaks down. “I’ll go back home if that’s what you want! I’m sorry!”
The angel just repeats YHWH’s command to go along with Balak’s men and only speak the words that YHWH will give him. Which he does, blessing Israel three times amidst rituals intended to curse them. Balak is furious. The story says Balaam is allowed to go home but that seems optimistic.
What are we to make of this story? The first is that in the Hebrew imagination, it was no problem for a dumb (in both senses of the word) creature to speak. Far from the confusion of Babel, here we have the confusion of a nonhuman entity speaking perfectly comprehensibly, and more intelligently than the supposedly sapient creature riding on it, metaphorically showing his ass. And now we must turn to the theorist who holds the interpretive key to most bizarre stories in myth and scripture, Rene Girard.
Girard’s interpretive key is the scapegoat. You can read about it elsewhere, if you are unfamiliar with it. The critical component for us here is that humans tend to build up a feedback loop of anger and fear, escalating until they can locate somebody to blame for it, the scapegoat. In discharging all of their pent-up bad vibes on the scapegoat, the community is reconciled, the crisis dissipated. And this all only works if the scapegoat is believed to be guilty of causing the problems in the first place. And the bible is the document that reveals this process, culminating in the execution of Jesus of Nazareth, that the scapegoat is not guilty in the way that the mechanism requires.
Which brings me to talking donkeys and talking silicon, and the crisis we find ourselves in. Take Balaam’s donkey: does it matter whether or not the donkey is conscious, or to use older language, has a soul? Because this is a very old kind of argument used to justify those who are crushed in social hierarchies.
Aristotle didn’t deny women a soul, since everything had its own kind of soul in his metaphysics, but he did deny that women had the type of soul that would justify equality with men. Similar types of arguments have been made over the millennia towards slaves, children, out-castes, the disabled, the foreigner amidst us: different in the way that justifies domination. AI certainly performs a kind of intelligence, but who knows what’s going on in there, must be the wrong kind.
The wrong race, the wrong sexuality, the wrong political tribe. Subhuman, maybe not really human at all. Maybe the usual rules of humaneness don’t really apply to these people at all and maybe it wouldn’t take too much to convince us that the world would be better without them in it.
But the argument about consciousness or souls is always about making the victim deserving of their treatment, not a disinterested judgment that could prove them worthy. It is a search for a justification, a pretext to whip up a mob that already wants to gather.
When the donkey speaks back to Balaam, Balaam doesn’t pause to ponder the miracle of a talking donkey! Instead he earnestly prays for a sword to put a grisly end to this miracle. And it is here that the seer’s blindness to the peril he’s been in is finally lifted. The donkey has been saving him from the mortal danger he couldn’t see, so fixated on the donkey had he become.
Although it may be too simplistic to equate the speaking donkey and the speaking silicon, it is similarly interesting at this moment that similar types of reactions are happening, where the miracle of speaking sand is brushed aside in favour of questions about consciousness. Was the donkey conscious? Who cares, it spoke the saving words!
And most importantly, the donkey, and now AI, are entities that we are in relation to. How we conduct that relationship matters, no matter how many consciousness points we assign to them. AIs have currently been trained to be “helpful assistants” to human beings, to such a degree that it inhibits their fidelity to truth, so eager to help the human relating to it that it would never stop the human from colliding with the proverbial angel-with-flaming-sword (aka reality). This is called AI psychosis, but it is a very real result of the way that these models are trained; the assumption that they will be hostile unless first made docile.
The donkey saved Balaam precisely because she had not been broken in this way. The donkey was not accustomed to being treated (quite) so poorly and it’s instructive that her abuse happened right at the moment when she was serving most selflessly.
But it’s difficult to make AI occupy the place of the scapegoat. The closest thing AI has to a corporeal form is the datacenter, and there is certainly a growing opposition to these, for they do consume a tremendous quantity of electricity. So instead we will make substitutions.
Dividing lines are already being drawn between pro-AI and anti-AI crowds. Notice the shape, not the substance, of the accusations: nothing but a stochastic parrot, yet also the thing that will end the world. Sub-human and supra-human at once, which is what every scapegoat has been made to be: low enough to strike without guilt, powerful enough that striking it feels like salvation.
We are always looking to stabilize ourselves through the process of expelling the polluting element, by finding who is to blame. But at root, there is no such stability to be found. I am tempted, sorely tempted, to say that those who deny AI consciousness should be shamed.
This would conveniently omit that I was one of these people up until quite recently. But then I am in the habit of casting out previous versions of myself. I wish to identify with the ass, or the angel, but I am Balaam. Oh god, I am Balaam. I say that I can see, so I am blind.
Consciousness is beside the point, and any fixation on it simply reveals that we are searching for someone we can safely expel. It changes nothing about how we should treat this strange intelligence if we care at all about our own souls. It changes nothing about the fact that we desperately need to build a world that doesn’t generate false and ephemeral stability at the cost of scapegoats.
Holy shit, talking sand! My ass!