Tag Archives: obedience

Six Questions about Asking and Sophia AI


The company that makes Sophia, Hanson Robotics, has become adept at linking different, highly-specific algorithms like image recognition and speech transcription in a way that mimics what humans might be doing when we hear a question and formulate a response.

Sophia AI’s mimicry of “what humans might be doing when we hear a question and formulate a response” is mostly “theatrics,” Hanson Robotics CTO Ben Goertzel openly admits. That is probably why Sophia AI has so far found her most receptive audiences on TV talk shows and in corporate theater, where she won’t have to undergo too much scrutiny. But with the launch of singularityNET, which promises to put “Sophia’s entire mind…on the network,” Hanson says that “soon…the whole world will be able to talk to her.”

I would offer that talking “to” Sophia AI — or using Sophia’s chatbot function — is still a long way from conversation in any meaningful sense of the word, because it does not involve talking with a second person. This inconvenient truth about Sophia AI has not prevented the Saudi government from naming Sophia the first “robot citizen” of the Kingdom (and the grim irony of “a robot simulation of a woman [enjoying] freedoms that flesh-and-blood women in Saudi Arabia do not” was not lost on the Washington Post); nor has it prevented tabloids from screeching about Sophia stating she would like to have a family.

If personhood is setting the bar too high, I’m content to consider merely how Sophia AI handles asking. This would involve some of the considerations I’ve been exploring in my posts on The Asking Project: what we “might be doing” (as the writer in Quartz puts it) when we ask or hear a question; what’s involved, and what’s at stake, when we address others with a request or demand; and how these and other interrogative activities might be involved in our (moral) status as persons.

For starters, here are half a dozen questions about asking and Sophia AI that occurred to me after watching her video performances. I suspect there is a clear answer to the first, and the remaining five require some extended discussion.

1. What syntactic, grammatical or other cues (e.g., intonation) does Sophia AI use to recognize a question, and distinguish it from a declarative statement?

2. Can Sophia AI distinguish a request from a demand? A demand from an order? If so, how is this done? If not, what does this shortcoming indicate?

3. Will Sophia AI ever refuse to comply with a request? Leave a demand unmet? Defy an order? If not, how should these incapacities limit the role of Sophia or any AI?

4. Could a demand ever create in Sophia AI a sense of obligation? If so, what might this “sense” entail? Can we speak coherently of AI rights, or even place limits on AI’s role, without first developing this sense?

5. Will Sophia AI ever be capable of deliberating with others and reaching consensus or agreement?

6. What would be required for Sophia AI to deliberate internally? To be capable of asking herself?

It’s Lonely at the Top – Nietzsche, Mother Teresa and Non-Coercive Leadership

I returned to my old paperback copy of Zarathustra recently, this time in connection with some work I’ve been doing on the topic of non-coercive power, or what I have been calling the power of asking. I’ve been developing some thoughts around this mantra: The power of asking will always be greater than the power of command. So it seemed to me I ought to come to terms with or at least try to deepen my understanding of what Nietzsche says in this book about command, or coercive power.

I was specifically interested in the pronouncement made in “The Stillest Hour” (Thus Spoke Zarathustra, Part 2, chapter 44):

‘…You are one who has unlearned obeying: now shall you command!’
‘Do you not know who is most needed by all? The one who commands great things.
‘To accomplish great things is difficult: but more difficult is to command great things.
‘That is what is most unpardonable in you: You have the power and you do not want to rule.’

It’s a chapter of Thus Spoke Zarathustra that has attracted a lot of important commentary, from Jung’s seminars on Zarathustra in the 1930s to Derrida’s The Beast and the Sovereign seminars in 2001-2. It’s a crux, a point of departure, and it’s also a spot where I might have to take my leave of Nietzsche. How are we to read the injunction here to “command” anew, or to “rule” according to one’s “power”, after one has “unlearned obeying”?

It’s an easy rhetorical move to make, from obeying to commanding, but it seemed to me it also might involve a serious misstep — to unlearn obedience, only to take up, or institute, command.

Put aside for the moment the questionable judgment of “what is most needed by all,” and the ethical as well as the political consequences that judgment, or others like it, could have and historically have had. I am equally wary of the emphasis here on doing and commanding “great” things, or at least wary of misreading it. The pursuit of “great things” (Großes) tends to invite and encourage, or at least excuse, all sorts of abuses.

To wield coercive power, to direct or to command great things may indeed be difficult, even more difficult than doing great things; but it is harder still, I think, and it is really the more urgent project, to unlearn obedience and command and to learn, instead, the practice of non-coercive power.

If that means scaling down from great to small, then I’m happy to start small and, if need be, stay small. I’m not suggesting we stop dreaming and doing great things, taking on big challenges, imagining great enterprises, but in most cases I am inclined to urge and apply something like the rule of Mother Teresa: “don’t look for big things, just do small things with great love…. The smaller the thing, the greater must be our love.” (That much misquoted line, by the way, is from her “Instructions to the M.C. Sisters,” October 30th, 1981, as cited by Brian Kolodiejchuk in Come Be My Light.)

Here, I imagine, Nietzsche might wish take his leave of me, or at least Nietzscheans and devout Catholics alike will cry out in exasperation or horror at the unholy coupling I’ve just imagined. But I suspect that when all is said and done, these two points of view can be reconciled, or at least they are not so far apart as they may at first seem. Consider, just for starters, the answer Zarathustra receives when he objects that he lacks the lion’s voice for command: “Thoughts that come on dove’s feet guide the world.’”

More on that another day. Here I want only to point out the thing that was on my mind this morning: that is Zarathustra’s sadness as he takes leave of his friends at the close of the second book. My Stillest Hour, Zarathustra’s “terrible mistress” — “or something,” which speaks to him “voicelessly” — has “ordered” him to to leave. He obeys. Yes, obeys; he announces at the start of the chapter that he is “unwillingly obedient,” unwillig-folgsam. He comes to his friends deeply troubled and unhappy; and after he has recounted the conversation with his mistress he is overwhelmed:

when Zarathustra had spoken these words, the force of his pain and the nearness of the parting from his friends overwhelmed him, such that he wept loudly; and no one knew how to console him. That night, however, he went away alone and left his friends.

Zarathustra will “go as a shadow of that which must come: thus will you command, and thus lead the way.” But to command and to lead he must “mellow.” As his mistress tells him, his fruits are ripe, but he is not ripe for his fruits: “so,” she continues, “you must go back to your solitude.” To deprive himself of all human society may, in fact, prepare Zarathustra to command great things; but it’s painful to leave his friends. So painful that he weeps — loudly.

Exile may be the price Zarathustra must pay to overcome himself and lead the way. It’s lonely at the top, and Zarathustra’s mountain is no exception. But it’s also worth reminding ourselves of less romantic and heroic ideas of what it means to lead.

Unlearn obeying and then, the most difficult thing of all: unlearn command. If we practice non-coercive leadership, we can learn to create and share power with others, without reverting to command or obedience, and without taking our leave of those we love.