Will A.I. Overwrite Our Sense of Shame?

The religious terminology I am using here is intentional. If religion has traditionally been one of the chief mechanisms for enforcing shame, LessWrong and its posters have flipped many of the conventions once regulated by shame on their head. The nuclear family has been replaced by the polycule, which many of these posters believe is a more rational arrangement for sexual relationships; the world’s competing moral philosophies have been narrowed down to the supposedly objective value of “optimized good”; the divine has been replaced by the god we create through our logic, expressed, at least when it comes to A.I., using code.

The tech world has always had a self-consciously rebellious relationship with societal norms. The industry’s titans, from Steve Jobs to Elon Musk and Mark Zuckerberg, have pitched themselves as iconoclasts willing to follow the early Facebook credo: Move fast and break things. The alleged virtue of shamelessness has always been a part of this act, although the revolt has not usually extended much beyond, say, Zuckerberg’s donning a hoodie instead of the suit that society allegedly expected him to wear. The nerd, dissatisfied with a culture that once rejected him in favor of the Chad, now gets to dictate who does what and goes where.

This defiance can be thought of as a refusal to feel shame—or, at least, as the confidence to believe that the shamers are wrong. In December, 2src2src, Sam Altman wrote, on his personal blog:

The most impressive people I know care a lot about what people think, even people whose opinions they really shouldn’t value (a surprising numbers of them do something like keeping a folder of screenshots of tweets from haters). But what makes them unusual is that they generally care about other people’s opinions on a very long time horizon—as long as the history books get it right, they take some pride in letting the newspapers get it wrong.

In other words, as long as you are confident that you are right and the sheep are wrong, you can skip feeling bad about public opinion, because the historians will ultimately absolve you. If today’s shame structure makes you feel bad about your choices, just zoom out and think of yourself as Gandhi among the Brits.

Meanwhile, Yudkowsky took such tropes of rebellion and rule-breaking and built a philosophical apparatus around them. Nearly everyone who has worked on A.I. alignment—the process attempting to insure that A.I. works in a virtuous manner, in accord with basic ethical principles, and doesn’t build a bioweapon or turn us all into paper clips—has at least a passing familiarity with his work. This was likely even more true of those who worked on today’s leading A.I. programs in their earlier days, when those models first started to distinguish right from wrong.

If you think of alignment as the way that humans upload shame onto A.I., then it seems reasonable to worry that the robots beginning to infiltrate our daily lives may reflect the social norms of a handful of analytic philosophers and A.I.-safety engineers—and the posters of LessWrong—far more than they reflect those of the rest of society. These values might all be coming from smart and thoughtful people, but they tend to be smart in the same way; they rebel against the same norms; they largely uphold the same values. The majority of Americans, according to pretty much every recent poll, do not trust the machine brain they birthed, or the people behind it.

Of all the issues associated with a doomsday scenario for A.I., the one I find myself most concerned about is the potential to short-circuit our collective sense of shame. If we accept that A.I. chatbots are on their way to replacing much of our current information technology, and if we, as good McLuhanites, believe that the medium is indeed the message, then we have to ask whether the entire world will soon yield to Yudkowskian philosophy. We have seen cultural transformations follow the advent of the printing press and the television, as our lives and our politics were shaped by a new machine. But has any machine itself been shaped by such a narrow and specific philosophy? At the very least, there has never been a machine that can tell you all about its philosophy, in the soothing simulated voice of your choice.