Why China Isn’t Getting Existential About A.I.

But, more recently, you have a couple of new major A.I. risks showing up at higher and higher levels of the Chinese Party state. And one is cyber risk. Mythos was a wake-up call for Chinese policymakers. Mythos is the model that Anthropic had developed and announced in April this year, and it appeared so powerful in terms of cyberattack capabilities, and being able to find vulnerabilities in widely used software platforms, and chain them together in a very sophisticated way that basically matched, or even maybe exceeded, teams of very, very strong hackers.

Anthropic withheld the release of Mythos until it could share the new model with a select group of government entities and companies first. China was watching all this very closely. And Mythos sent shock waves through the Chinese policy establishment because it signalled that the potential for A.I. to be used for cyberattack and cyber defense had entered a whole new era. And it’s been talked about in Chinese state media, and then, most recently, we had the head of China’s Ministry of State Security calling out, specifically, Mythos, and a similar model from OpenAI for heralding a new era of A.I.-enhanced cyberattack capabilities, and what that could mean for China’s national security.

And then there is the loss of control of some almost runaway A.I. systems. China’s been following very, very closely what’s happened with the Hugging Face incident, in which an OpenAI model broke out of its sandbox where it was supposed to be contained for testing within an environment that was not connected directly to the internet. And it was supposed to be tested safely within the confines of that kind of cyber cage. But what happened was that it got out, and it attacked an American A.I. platform called Hugging Face. And perhaps the most alarming part is that OpenAI did not know this was happening until later on. So this is an example of A.I. itself getting out of control and potentially causing real damage. In this case, there wasn’t actually any economic damage, but you can see that it wouldn’t take a huge leap to see the kind of damage that could happen if it weren’t Hugging Face that were attacked but, say, some major digital infrastructure platform in the U.S. or China. Those incidents have also been written about a lot in Chinese state media. On the one hand, they try to frame it as, at least publicly, a kind of American issue. And, ironically, Hugging Face used a Chinese model to defend themselves, so in some ways China tried to portray this as a victory for Chinese open-source A.I. models. But you can see in other statements, including from the Cyberspace Administration of China, that they’re worried about these sorts of, what they call, “loss-of-control incidents,” where the technology gets out of hand, where the A.I. system can operate autonomously, and humans can’t even track what it’s doing. And it does things that were not what was originally intended by the human operators.

The Hugging Face incident set off all these concerns in America about existential-risk scenarios as well. But it seems like from what you’re saying that the response to the incident was different.

Yeah, so in the U.S. we also have these emerging loss-of-control concerns about A.I. systems with their current capabilities getting out of hand, potentially hacking other websites, hacking companies, doing damage at that level. But there’s another level, which is if we can get A.I. systems to improve themselves, to create this feedback loop of continuous self-improvement, where they get better and better and better, the worry in the U.S., at least among some folks in the A.I. community, is that this trend will accelerate to a point where humans no longer have control over the process of A.I. self-improvement. And then the A.I. will potentially become its own sort of autonomous entity, potentially with its own goals, or its own decision-making processes. And then the most scary part in that vision is that it would be super intelligent, and it would be able to hack and infiltrate digital systems and come up with novel biological pathogens and produce all of them in a way that we can’t even imagine. And this is where it starts to enter into a kind of sci-fi-type scenario, but that’s the genuine belief of many who are concerned in the U.S. A.I.-safety community.

How much do you think the C.C.P. views A.I. as specifically a threat to their rule?

One thing they’re worried about is the U.S. using some of this technology to undermine C.C.P. control. And I’m drawing here from the specific piece written by the Minister of State Security, and it focussed on deepfakes and efforts to manipulate what the C.C.P. considers China’s ideological space. That is something that they’re very worried about, and they’re worried that other so-called foreign actors might try to use A.I. to influence the social and political fabric in China. And that could be a direct challenge to the C.C.P.’s control over the country.

What do we know about the degree to which Chinese citizens are concerned about A.I.?

People are concerned in two ways. One is they’re worried that they’re not running fast enough with A.I. They’re worried that, given the already intense competition for jobs, especially among more college-educated Chinese people, especially for the kinds of jobs that they think are commensurate with their level of education, there’s a big supply-and-demand mismatch. And there’s been growing and growing anxiety about being one of the lucky few who can nab a good job. This is part of this broader term that was originally used in the Chinese context for the labor market, which is called involution. And it’s this idea that competition is so intense and so fierce that you, as an individual, feel like you are running faster and faster. But you are actually still falling behind, and not able to make progress. That’s the kind of anxiety that people feel in China, even before A.I. And then what A.I. did was kind of turbocharge that anxiety, where people felt like they needed to quickly adopt and use it and learn how to leverage A.I. to make themselves more competitive in the job market. So that’s one side of it.

The second concern is just outright job displacement. This is becoming a greater and greater issue among people in China, and it depends on your job type and where you sit in the income or educational structure, but this is a growing concern. And I don’t think it’s reached the same kind of level that you see with, say, Gen Z feeling like they’re cut out of the career ladder in the U.S. But you can break it down, sector by sector, whether you’re talking about white-collar workers, especially young white-collar workers, or salesfolks. And then, even in the physical world, factory workers and delivery drivers are worried about greater A.I. autonomous delivery, for example. And that’s growing, even though, for a while, the issues seem to almost have been downplayed in the official media discourse.