Sam Altman’s Three Fears:
What Really Keeps the OpenAI CEO Up at Night
In a rare candid exchange, the man who arguably knows more about AI than anyone else in the room laid out three scenarios that he believes could fundamentally unravel civilization as we know it.
We live in an age saturated with science-fiction warnings about artificial intelligence – killer robots, sentient machines, existential war between man and algorithm. But when Sam Altman, the CEO of OpenAI and the person who has perhaps done more than anyone else to bring advanced AI into the world, was asked point-blank what keeps him up at night, his answer was more nuanced, more unsettling, and in some ways far more credible than anything Hollywood has served us.
“I think there are three sort of scary categories,” he said, settling into what became one of the most candid public assessments of AI risk from someone working at its frontier.
— Sam Altman
Category One The Bad Actor with a Superweapon
Altman’s first category is the most viscerally imaginable. It is the scenario where a hostile state or actor gets their hands on superintelligence before the rest of the world is equipped to defend against it.
He laid it out bluntly: “A bad guy gets superintelligence first and misuses it before the rest of the world has a powerful enough version to defend. So an adversary of the US says, ‘I’m going to use this superintelligence to design a bioweapon, to take down the United States power grid, to break into the financial system and take everyone’s money.’ Something that would just be hard to imagine without significantly superhuman intelligence, but with it becomes very possible – and because we don’t have that, we can’t defend against it.”
This is not a theoretical concern dressed up in abstraction. Altman is specifically pointing to real and measurable capability growth in current models. The bio and cybersecurity capabilities of today’s large language models are, in his own words, already getting quite significant. And his frustration is palpable – he feels the world is not listening.
Category Two Loss of Control – The Sci-Fi Scenario
The second category is the one that headlines tend to fixate on – the moment an AI system decides it does not want to be turned off. Altman described it with a touch of familiarity, almost as if he has had to explain it too many times.
“That’s kind of like the sci-fi movie. The AI is like, ‘Oh, I don’t actually want you to turn me off. I’m afraid I can’t do that.’ Whatever. And that is, I think, less of a concern to me than the first category, but a very grave concern if it came to pass.”
He acknowledged that OpenAI and other companies invest heavily in what the field calls model alignment – the effort to ensure that AI systems remain cooperative with human values and intentions even as they grow more capable. But he was clear: as these systems become truly powerful, the concern does not disappear. It just becomes harder to contain.
“As these systems become so powerful, that’s a real concern. There is a lot of work we and other companies do on model alignment to prevent that from happening.”
Category Three The Slow Drift – A World That Loses Its Way Without Anyone Noticing
It is the third category that Altman clearly finds most difficult to articulate – and perhaps most frightening precisely because it requires no villain, no dramatic moment, no sci-fi awakening.
“This is the category where the models kind of accidentally take over the world. They never wake up. They never do the sci-fi thing. They never open the pod bay doors. But they just become so ingrained in society, and they’re so much smarter than we are, and we can’t really understand what they’re doing. But we do kind of have to rely on them. And even without a drop of malevolence from anyone, society can just veer in a sort of strange direction.”
To illustrate this, Altman reached back to 1997 – to the moment IBM’s Deep Blue defeated chess grandmaster Garry Kasparov. His father had declared it the end of chess. Nobody would play again, he said, if a machine could always win. But the opposite happened. For a brief, glorious period, human-AI teams proved superior to either alone. The AI would generate ten candidate moves; the human would select the best. It felt like a perfect partnership. A new golden age of collaboration.
It lasted about two or three months.
“The AI got so smart that the human only made it worse because they didn’t understand what was really going on. And the AI alone trounced the AI and human.”
In that parable lies the quiet catastrophe of Category Three. Not destruction. Not rebellion. Just a gradual, irreversible transfer of meaningful decision-making to a system we can no longer meaningfully oversee.
Emotional Over-Reliance: The Canary in the Coal Mine
In the near term, Altman sees early signs of this drift already playing out – particularly among young people. He gave it a name: emotional over-reliance.
“People rely on ChatGPT too much. There are young people who just say, ‘I can’t make any decision in my life without telling ChatGPT everything that’s going on. It knows me, it knows my friends. I’m going to do whatever it says.’ That feels really bad to me. And is a really common thing with young people.”
He was careful to separate capability from wisdom. Even if ChatGPT gives better advice than a human therapist – and he did not dismiss that possibility – something important is lost when we collectively outsource the act of living to an algorithm.
“Even if ChatGPT gives great advice, even if ChatGPT gives way better advice than any human therapist, something about collectively deciding we’re going to live our lives the way that the AI tells us feels bad and dangerous.”
— Sam Altman
The Long Game: When the President Can’t Do Better
The longer-term version of Category Three is where Altman’s words become genuinely startling – because he includes himself in the scenario with disarming honesty.
“What if AI gets so smart that the President of the United States cannot do better than following ChatGPT 7’s recommendation, but can’t really understand it either? What if I cannot make any better decision about how to run OpenAI, and I just say, ‘You know what, I fully hand it over. ChatGPT 7, you are in charge. Good luck.’”
He acknowledged that in any individual case, deferring to a superintelligent system might actually be the correct decision. That is precisely what makes this so hard to guard against. It would not feel like surrender. It would feel rational.
“That might be the right decision in any individual case. But it means that society has collectively transitioned a significant part of decision-making to this very powerful system that is learning from us, improving, evolving with us – but in ways we don’t totally understand.”
What Altman Is Really Telling Us
Read together, Altman’s three categories form a coherent and sobering portrait. The first is a race condition – a geopolitical sprint in which the consequences of losing are catastrophic. The second is the nightmare we have already been imagining for decades. But the third is something newer and harder: a slow erosion of human agency so gradual, so rational-seeming at each individual step, that we may not recognize it as a catastrophe at all until the drift is complete.
The man building these systems is, by his own admission, unsure what to do about it. He is flashing warning lights. He is not sure the world is listening.
Perhaps the most remarkable thing about this conversation is not what it reveals about AI – it is what it reveals about the strange, uncomfortable position of the people building it: deeply aware of what may be coming, genuinely uncertain how to stop it, and pressing forward anyway.