Book review: Karen Hao - Empire of AI

For Empire of AI: Dreams and Nightmares in Sam Altman’s OpenAI, Karen Hao conducted 300 interviews with around 260 people and scrutinized countless documents and correspondence. The result is an impressive and comprehensive account of an AI race that OpenAI didn’t want, but did set in motion.

Sam Altman

It is obvious from page one that the central figure in this book is Sam Altman, the Stanford dropout and co-founder of OpenAI. The book paints a picture of Sam as a savvy dealmaker and investor. An entrepreneur who, before OpenAI, co-founded the not-that-successful location-sharing startup Loopt and then joined startup accelerator Y Combinator. A man whose most important mentors were Paul Graham and Peter Thiel, the co-founder of Palantir. A dreamer who seriously explored the idea of running for governor of California. A brother who was accused of various kinds of abuse by his own sister, Annie, and would let her struggle in poverty while boasting a net worth in the billions. Someone who would tell lies and tell everyone exactly what they wanted to hear. A founder who would win a power struggle with Elon Musk over being the sole leader of OpenAI, only to then get fired as its CEO (an event internally referred to as “The Blip”). And a leader who, despite his many flaws, would get so many employees to protest that he was rehired, including the very people who wanted him gone.

While a book could certainly be written just about Sam Altman alone, thankfully, Hao uses this work to paint a much bigger picture of the world of AI, and the people and companies working to build and promote it at the expense of others.

A consequential dinner

In the early chapters of the book, Hao describes a pivotal moment: a dinner where Sam Altman and Elon Musk meet with Ilya Sutskever and Greg Brockman. The plan (or, the dream): to build AGI before anyone else does. Because wouldn’t it be better if we are the first to have something that dangerous? The gentlemen (and yes, it becomes clear that the empire consists almost exclusively of men!) would soon agree on the founding of OpenAI, an open and transparent non-profit entity to benefit “humanity.” Hao uses this moment to make it clear that OpenAI, from the very start, was about egos and power.

Hao uses this moment to make it clear that OpenAI, from the very start, was about egos and power.

Existential threat

The fear of an AI so powerful and dangerous that it may become an existential threat to humankind is a central theme throughout the book. An AI so strong that it deserves its own names and abbreviations: Artificial General Intelligence (AGI), or for those who want it stronger, Artificial Superintelligence (ASI). The fact that there isn’t a workable definition for either of these terms in the field doesn’t seem to bother anyone – although Sam would apologize to The New York Times for his continued use of the term, admitting that it is meaningless, just two days before being fired from OpenAI. He would continue to use it anyway.

Doomers and Boomers

Since the fear of AGI was so central to the founding of OpenAI, it may be no surprise that the company attracted many so-called "Doomers": those constantly on the lookout for signs of a (too) powerful AI, advocating for safeguards, safety research, and slowing down the pace of AI research and its release. Hao describes how this group intersects with the Effective Altruism (EA) movement that gained popularity in Silicon Valley around the same time, and also refers to parts of the group as the “safety clan.” There were also the "Boomers": those in favor of building and scaling up AI systems using unprecedented amounts of data and compute power, and of releasing them to the public. This group, too, was a dominant factor at OpenAI, and would gain more and more influence as money poured in after Microsoft’s investment.

Drama and incidents

While the Doomers vs. Boomers caricature might not fully do justice to reality (they are two sides of the same coin, really, and many researchers don’t subscribe strictly to either point of view), it is a useful one for Hao to describe the ideological divide both in the wider research community and inside OpenAI. A divide that was at the center of a lot of drama at the company, and of the many nightmares referred to in the subtitle. For example, Hao spends time describing the clumsy release of GPT-2: OpenAI’s policy team, leaning into the fears around AGI and foreign adversaries, and influenced by the so-called “safety clan,” thought the bigger GPT-2 models were too “dangerous” to release, and provided example outputs to prove their points. While academics back then definitely thought the GPT-2 models were unusually large language models and therefore potentially harmful, there was almost immediate criticism of the release strategy and the drama surrounding it. Hao describes how a Stanford professor mockingly asked Alec Radford, the lead author of the initial GPT model: “But is it dangerous?”

Hao describes how a Stanford professor mockingly asked Alec Radford, the lead author of the initial GPT model: “But is it dangerous?”

As we read on, it becomes clear that the GPT-2 release drama was merely the least dramatic incident in OpenAI’s history since Musk left with his funding. It was followed by the exodus of researchers to Anthropic in 2021, the release of ChatGPT (2022), the firing of Sam Altman by the board for not being “consistently candid” (2023), his reinstatement five days later, the NDA scandal (2024; OpenAI made departing researchers sign a contract stating they would not speak negatively of the company or otherwise lose all their vested equity), the GPT-4o Scarlett Johansson voice controversy (2024), the departure of co-founder Ilya Sutskever and safety researchers like Jan Leike (2024), and the restructuring as a for-profit public benefit corporation.

Data and Exploitation

It must be clear by now that just these controversies were enough for Hao to write a Michael Wolff-style book about OpenAI. Luckily, she goes much deeper than that, taking us to countries like Venezuela and Kenya that, in the midst of economic crises, were the “perfect” targets for the exploitation of low-wage workers to fulfill the AI industry's growing need for annotated data. It is especially commendable how Hao brings the perspective of these workers into focus, showing how they allow their computers to wake them up at night to take on new annotation work just so they can pay the bills. Apart from the low pay, Hao spends time discussing the psychological consequences of dealing with sensitive and upsetting data, and the seemingly little regard the AI world has for the lives of those who do the data work, discarding them (as in the case of Venezuela) when deemed no longer necessary.

It must be clear by now that just these controversies were enough for Hao to write a Michael Wolff-style book about OpenAI. Luckily, she goes much deeper than that, taking us to countries like Venezuela and Kenya that, in the midst of economic crises, were the “perfect” targets for the exploitation of low-wage workers to fulfill the AI industry's growing need for annotated data.

Separate from the data workers, a lot of room is given to discussing the rapidly growing number of data centers being built to train and serve AI models. Hao describes how tech companies build these centers in places where land is cheap but water is scarce, depleting local communities of a resource that is sometimes already at a critical level. It also becomes clear that the local community hardly benefits when a data center comes to their area. A notable example is an “urban forest” in Chile that is located in a spot so distant, and of a size so small, that it hardly benefits anyone there, serving just the purpose of justifying the data center and feeding the narrative of giving back to the community.

The book also pays attention to those who have critiqued these exploitative practices, such as Emily Bender, Alex Hanna, Timnit Gebru, and Meg Mitchell (co-authors of the “Stochastic Parrots” paper that would end up getting the latter two fired from Google). Indeed, it is in these more critical chapters that we get a break from the “tech bro” narratives around OpenAI, and see a different way to look at AI and what it can do. In one of the final chapters, Hao highlights the work of Te Hiku Media, which worked with the Māori community to build language resources for them – and with their explicit consent – to help keep the te reo language alive. The story makes for a stark contrast with the data-hungry, extractive practices of OpenAI.

Mission creep

It is late in the book where Hao is perhaps at her best, eloquently chronicling the remarkable mission creep of OpenAI: from its start as a nonprofit “unconstrained by a need to generate financial return” (2015), to “everyone should benefit from AI, but we don’t need to share the science” (2016), to a capped for-profit structure to obtain more financial resources and avoid an AI race (2018, 2019), to walling off their AI and providing it through APIs (2020), to “iterative development” and racing to deploy ChatGPT (2022), and finally to putting very capable AI tools into the hands of people for free, “or at a great price” (2024). The list shows how little of the initial dream of OpenAI is still left, if anything. In order to keep scaling up AI models, with their ever-increasing data and compute needs, OpenAI required billions of dollars in investments. Slowly, but then quickly, the resulting commercialization process gave the Boomers the upper hand.

The list shows how little of the initial dream of OpenAI is still left, if anything.

The fall of the empire

In the final pages of the book, Hao discusses a talk by Ria Kalluri, who proposed an alternative to the question of how to do “good” with AI. What is “good” is in the eye of the beholder, she says, proposing instead to talk about how AI “shifts power.” Power consists of knowledge, resources, and influence, and OpenAI has control of each of them in ways that reinforce one another. Labor protections, funding of independent research, policies – for example, regarding the disclosure of training data and sustainability – and education can help shift power back to communities.

What is “good” is in the eye of the beholder, she says, proposing instead to talk about how AI “shifts power.”

After Sam’s reinstatement, now without the fear of being fired by the now-dissolved nonprofit board, he seems to have gained more power than ever. ChatGPT was an instant success and has gained hundreds of millions of users. Yet, its popularity comes at a cost. OpenAI is losing billions of dollars just to keep operating. According to the FT, it needs to raise “at least $207bn by 2030 so it can continue to lose money.” Despite the dream and promise of AGI, the future of OpenAI is not at all certain.

And finally, what about AGI? No one knows what it is. If it is meant to be (human-like) cognition, we can find ample research claiming that such a thing is intractable. With that in mind: if we assume that AGI is not at all around the corner, and that current methods will not get us there, then what was all the fuss at OpenAI really about?

Altman’s startup Loopt ended up being acquired. It ceased operating, and no one ever heard of it again. The book leaves us wondering whether, when all the hype is over, OpenAI will suffer a similar fate.