Tag Archives: orders

Six Questions about Asking and Sophia AI


The company that makes Sophia, Hanson Robotics, has become adept at linking different, highly-specific algorithms like image recognition and speech transcription in a way that mimics what humans might be doing when we hear a question and formulate a response.

Sophia AI’s mimicry of “what humans might be doing when we hear a question and formulate a response” is mostly “theatrics,” Hanson Robotics CTO Ben Goertzel openly admits. That is probably why Sophia AI has so far found her most receptive audiences on TV talk shows and in corporate theater, where she won’t have to undergo too much scrutiny. But with the launch of singularityNET, which promises to put “Sophia’s entire mind…on the network,” Hanson says that “soon…the whole world will be able to talk to her.”

I would offer that talking “to” Sophia AI — or using Sophia’s chatbot function — is still a long way from conversation in any meaningful sense of the word, because it does not involve talking with a second person. This inconvenient truth about Sophia AI has not prevented the Saudi government from naming Sophia the first “robot citizen” of the Kingdom (and the grim irony of “a robot simulation of a woman [enjoying] freedoms that flesh-and-blood women in Saudi Arabia do not” was not lost on the Washington Post); nor has it prevented tabloids from screeching about Sophia stating she would like to have a family.

If personhood is setting the bar too high, I’m content to consider merely how Sophia AI handles asking. This would involve some of the considerations I’ve been exploring in my posts on The Asking Project: what we “might be doing” (as the writer in Quartz puts it) when we ask or hear a question; what’s involved, and what’s at stake, when we address others with a request or demand; and how these and other interrogative activities might be involved in our (moral) status as persons.

For starters, here are half a dozen questions about asking and Sophia AI that occurred to me after watching her video performances. I suspect there is a clear answer to the first, and the remaining five require some extended discussion.

1. What syntactic, grammatical or other cues (e.g., intonation) does Sophia AI use to recognize a question, and distinguish it from a declarative statement?

2. Can Sophia AI distinguish a request from a demand? A demand from an order? If so, how is this done? If not, what does this shortcoming indicate?

3. Will Sophia AI ever refuse to comply with a request? Leave a demand unmet? Defy an order? If not, how should these incapacities limit the role of Sophia or any AI?

4. Could a demand ever create in Sophia AI a sense of obligation? If so, what might this “sense” entail? Can we speak coherently of AI rights, or even place limits on AI’s role, without first developing this sense?

5. Will Sophia AI ever be capable of deliberating with others and reaching consensus or agreement?

6. What would be required for Sophia AI to deliberate internally? To be capable of asking herself?

Another Abuse of Asking

I’ve been interested for a while now in the way asking works: what exactly are we doing when we ask what to do (what we should or ought to do) and when we ask things of each other (when we make requests or demands)? For the most part, my posts on what I’ve called, for better or worse, the power of asking have focused on the abuse of asking, the confusion of asking with orders that are not open to deliberation, or the issuing of commands in the guise of requests, as when people use the nominative “ask” but aren’t asking anything at all.

When I talk about “abuse” in this context I mean, for starters, that these confusions and ruses and other kinds of indirection make asking an “act professed but hollow,” as Austin puts it in How To Do Things With Words. In Austin’s scheme, abuse is just one kind of infelicity, and I don’t want to be too strict about it, or pretend that the term covers all the instances in which asking does not come off as it should; but I’m drawn to talking about abuses of asking in part because I think it’s important to point out that acts professed but hollow may not only be insincere but also lack moral seriousness, in the sense that moral seriousness requires taking others seriously, giving them moral standing as second persons to whom one is accountable and answerable.

Abuses might take the form of a well-meaning effort to soften commands, so that people don’t feel pushed around or ordered about. That might seem like a harmless management ploy. But to allow that this professed asking is really a nicer way of commanding is to admit that abuses of asking can also mask real power relations. They don’t afford interlocutors equal standing or a share in power, or even the freedom to answer “no,” as genuine deliberation or serious conversation about what to do might.

An illustration is provided by what North Dakota Governor Doug Burgum said just this Wednesday past, before the police moved into the Oceti Sakowin Camp:

Our big ask for tomorrow is that, you know, anybody that’s remaining in the camp, we want to make sure they know that they have an opportunity to voluntarily leave, take your belongings, remove anything that you think might be culturally significant, and we’ll help you get on your way if you need to do that.

This “big ask” followed an eviction order issued by the Army Corps of Engineers that was backed by heavily armed, militarized police. Those who did not take the “opportunity to voluntarily leave” were arrested and forcibly removed. Governor Burgum might have chosen to present what is essentially an ultimatum as a request in order to seem fair and reasonable, or to defuse a tense situation. But it’s curious — isn’t it? — that it wasn’t just the governor who indulged this bureaucratic habit of speech. The governor’s spokesperson later made the same request: “We ask those that are remaining to pack up their belongings, to take off,” he said, and again repeated the offer to “help” with transportation.

Neither he nor the governor wanted to be giving orders, apparently. If their statements on this occasion can be set down as abuses of asking, that abuse is hardly the worst charge to be leveled against the governor of North Dakota in this situation. And to parse Governor Burgum’s language or that of his spokesperson on this occasion probably isn’t the best place to start reflecting on all that just went down at Standing Rock. But it’s important, I believe, to be look at what was said on this occasion and what was actually meant, how power presented itself and how it actually went about things.

The two are not even close.

The governor and his spokesperson were not asking anything at all. They were disguising not just an order but a threat of violence as a request, and publicly refusing to take responsibility for what might ensue. Apparently, they weren’t the ones giving the orders. By asking, or pretending to ask, they washed their hands of the situation. In essence, the governor said that it was up to the water protectors at the camp to keep the forces under the governor’s command from doing violence to them.

It is a classic example of the abuser’s refrain: don’t make me hurt you.

Serious Conversations, 7

In these notes on serious conversations, I keep circling back, it seems, to two ideas: first, that what makes a conversation serious is not its subject matter or tone, but the stance of its participants toward each other; and, second, that the conversational stance requires that we confer a certain authority on our interlocutors, or (to put it another way) recognize that they have standing to address us.

While other kinds of authority — title, rank, role — are of secondary importance, and can sometimes even get in the way, this moral authority or standing is fundamental. It does not have to be earned, proven or ratified by reference to some person, written instrument or record of accomplishment outside the conversation or by institutional set up. It is constituted and realized in the relationship you and I have — or, if that is just too clunky, let’s say it is the relationship you and I have; and it is sufficient authority for a serious conversation because it makes us mutually accountable to each other.

Where this equal human stature (or dignity) is respected (and appreciated), it can be a source of power: not just the power of one over another, but the power to make claims or demands of each other, or to ask and answer, and this power of asking is essential if we are going to deliberate in earnest about our situation or collaborate on something new.

The conversational stance allows for genuine co-creation, because it’s not founded on subordination or one person ordering the other about. And the capacity for co-creation, the creative power that we share, only increases as we include more people in the circle of the conversation. (Of course there are limits: the research on group size and social complexity Dunbar summarizes suggests the circle probably should not widen beyond 150 people.)

I’ve tried to capture this thought in a simple rule: the power of asking will always be greater than the power of command.

That’s the basic position.

Another way to put the same thought might be in terms of the mechanics of ordering versus asking: whereas in the former we have one person directing the will of another, as we might address a short-order cook, in the latter we direct each other’s wills, so that we are, to stick with the metaphor, chefs in our own kitchen.

Of course the usual caveat applies about too many cooks spoiling the broth, I guess, but let’s also remember that people have different talents, training and competencies, and we can worry about how to order and organize ourselves once it comes to the actual cooking. Right now we’re just having a conversation.

Let’s also acknowledge, while we’re at it, that short-order cooks are models of industrial-era efficiency (but no longer efficient enough for the post-industrial fast food kitchen); gains in co-creativity can and probably will translate to losses in short-term efficiency.

Some concessions on one side or the other will probably have to be made, but too often the proponents of efficiency win without any argument, and people start giving orders or setting out plans for what’s to be done before the conversation even has a chance to get started. That’s when all the real power goes out of the room.