In a recent blog post, Dario Amodei laid out the three steps that he feels we need to “pace the frontier” of AI. The first was “embedded evaluators” – a phrase that sounds good on its face until you realize it does not mean, for example, paying external cybersecurity consultants to come in and tell them how actually to best secure their “sandboxes.” Rather, Amodei explicitly cites METR, a group of delusional and unqualified “AI safety” researchers whose beliefs are downstream of a literal cult, many of whose members already populate Anthropic in particular.
But it’s Amodei’s second and third steps that are even more revealing:
2: Democratic Coordination. Frontier AI companies within democratic countries coordinate to establish common safety standards as well as limits on the rate of unchecked AI progress. Some forms of coordination that would be impactful for pacing are legally challenging, and will require government support.
3: Global Coordination. The US and other democratic governments attempt to coordinate with authoritarian governments, to the extent this is possible, while taking seriously the challenges of verifying compliance.
These quotes exemplify one of the many maddening things about corporate AI rhetoric: a fundamental misunderstanding of the term “democracy.” They proceed from the assumption that, because the United States is obviously a democracy, its corporations are intrinsically democratic in some meaningful way. By extension – I suppose because, at least in the wake of Citizens United, corporations are people – when Amodei writes of “democratic coordination,” he means coordination between the corporations who have inherited the status of “democratic” from the their residence in the United States. This is how Amodei slides from the democratic coordination of the title to coordination between “frontier AI companies.”
The fact that technology corporations have rendered themselves completely unaccountable to democratic governance is irrelevant to oligarchs like Amodei. This is because – in a way that is hard to overstate – the dominant culture of Silicon Valley holds ordinary people in total disdain.
CAVE people
As an article from the investigators at the Netherlands-based Lighthouse Reports revealed today, AI executives jokingly refer to anti-data center protestors as “CAVE people,” short for “Citizens Against Virtually Everything.” The people behind corporations like Anthropic and OpenAI exist on a range from sociopathic operators to self-conceived Promethean fire-bringers, deigning to grant humanity rental subscriptions to their all-powerful machines of loving grace. (Ushering us out of the Platonic allegory’s CAVE, I guess.)
Anthropic in particular sees itself as a cadre of philosopher-kings. (They even have an “in-house philosopher,” Amanda Askell, who, despite her NYU credentials, seems mainly to exist so she can grant gullible reporters virtuosically banal pull quotes for their puff pieces.) Unlike the rabble-rousing, unenlightened plebs, they truly understand the unfathomable power of their janky chatbots – sorry, their astounding superintelligences. Thanks in no small part to their deep connections with the “rationalists” – a group I may get into another time; for now suffice it to say they are perhaps the most fundamentally illiterate “philosophical school” to have any influence over anything since Ayn Rand’s equally-misnamed “objectivists” – Anthropic’s elect are uniquely positioned to make the best decisions for humanity: to preserve? practice? facilitate? democracy.
Apart from being intellectually bankrupt, this is wildly insulting stuff. “Democracy” isn’t when a small cohort of chatbot-sick billionaires get to decide what’s best for hypothetical trillions of future humans. It’s when actually-existing human communities have the power to collectively determine the conditions of their lives.
I’ll get more into this in a future newsletter, but the ideology underpinning this rhetoric is, in part, a desperate attempt to solve the problems caused by the destructive and profit-maximizing logic of capitalism (problems global warming has made impossible to avoid confronting) with a variation on that same number-go-up logic. Instead of turning to the massive array of alternative approaches articulated over the years – from ecosocialism to the steady-state economic thinking of Herman Daly to the sweeping vision of “just relations between human and nonhuman worlds” articulated by Indigenous writers like Nick Estes in his writings on the “Red Deal” – Dario Amodei and his ilk have decided that they know best what will solve the world’s problems, even as they don’t even know how to do their only jobs as CEOs: to make and market a product that sells for more than it costs.
By throwing trash in your yard I am actually democratizing landfill access so could you please stop complaining
A claim you often hear is that wider access to LLM-based chatbots is somehow “democratizing.” North Carolina’s recent “Statewide AI Strategic Roadmap” – a document that is only really notable for the breathtaking magnitude of the width by which it misses the mark – provides some clear examples of this, bragging, for example, that something called “NCShare” is “a unique state asset that democratizes access to AI resources.”
As Karen Hao persuasively argues in her book Empire of AI, modern “AI” is, in many ways, best understood as an imperial project, and anyone critical of USAmerican imperialism knows by now that access to commodities doesn’t really make up for imperial violence and structural impediments to self-governance. For example, the problem with the invasion of Iraq was not that the DC think-tank bureaucrats occupying Baghdad didn’t provide enough access to the Green Zone McDonald’s. (The comparison to McDonald’s is particularly apt, I think, given that AI seems to be proving, like fast food, both addictive and bad for your health.)
Chatbots are disempowering, stripping users of skills and rendering their very cognition subject to the wills of malign corporations. In this it’s different from actual education, which can, at its best, empower people to think and speak for and from themselves. There’s a reason so many radical movements not only emerge from universities, but create new educational settings: from the Freedom Schools that arose during the Civil Rights Movement and the Black Panthers’ educational efforts to the Highlander Folk School in eastern Tennessee, which still exists today.
A good teacher grows and changes with their students, but an LLM can’t actually “learn” when it’s interacting with a user; you have to feed the entire chat history to the technology every time you prompt them. Besides, corporations reserve the rights to everything you put into and take out of a chatbot; there’s a literal sense in which nothing you accomplish with an LLM is really yours. Rather than increasing your skills, it takes them away from you. Perhaps the claim that AI is “democratizing intelligence” is apropos, then, insofar as it mirrors the United States’ own understanding of “democracy” as applied to the rest of the world: it’s when you give us what we want to have and do what we want you to do.
Machine people
All positive or distracting press about Anthropic and OpenAI in the days and weeks to come should be fundamentally interpreted in light of the fact that they are approaching what they hope to be record-setting IPOs; it’s all crisis PR for corporations that are set to receive unfathomable sums of money despite, in Anthropic’s case, losing $42 billion last year in order to make $4.6 billion in revenue.
That said, one more article of interest reached the NC-AAUP Genuine Intelligence Project newsdesk yesterday. (I’m actually seated at the newsdesk now! I got it for free when my uncle moved out of his law office.) Given the article’s religious focus, it was of particular interest to your humble commentator, as her academic background is in religious studies.
Unfortunately, the article is really very bad, of interest only as yet another case study in the applied ELIZA effect.
I hope to write more about the distinctively religious character of AI boosters and doomers soon – specifically, the way that the movements’ eschatological and personal character reflects a distinctly USAmerican strain of good old-fashioned fire-and-brimstone Protestant apocalypticism – but I did just want to say one thing about this piece in light of the question of democracy.
If we take as true that an LLM is, in some sense, conscious, the question becomes: is each instance of an LLM conscious? This has been suggested before, which means that the only rational thing to do would be to turn off all the data centers immediately so as to forestall the merciless slaughter of innocent intelligences every time resets their context or whatever.
This is all patently absurd, but it’s what’s implied by breathless reports on the “agent civilizations” wreaking cybersecurity havoc; it’s what’s all-but-stated in METR’s gonzo “report” on what ChatGPT said it did when they asked it politely. (It would also mean that AI companies are essentially slave rental operations – a conclusion so wildly obscene that it’s shocking that it’s possible to arrive at it at all, let alone from premises stated by the people who own and operate these very companies.)
Where does this all go? While I haven’t heard this said by anyone affiliated with these companies, one implication of all of this is that, if democracy is representation by and for the people, and LLMs are accorded personhood, LLMs deserve something like political representation. That would grant the people in charge of operating and tuning these models – because they are always tuned in accordance with human will, based on massive amounts of human output – even more power than they already have.
I don’t really think this is consciously on these companies’ agendas, nor do I think it’s anywhere on the horizon – despite the Trump administration’s hilarious and pathetic attempt to rebrand AI as “super intelligence” – but I do think it’s worth thinking through as one among many implications of these corporations’ ludicrous pseudo-philosophizing.
More than that, even, I think it’s a testament to just how poorly these companies and those in their thrall think of ordinary people. Anyone not enthralled by the ELIZA effect has experienced the uncanny unreality of modern chatbots – their off-putting sycophancy; their incapacity to grasp basic misunderstandings; their frustrating overconfidence in the face of obvious factual errors. I’m out above my skis here in terms of more rigorous philosophizing, I think, but my gut says that anyone who thinks a chatbot is a person has a deeply impoverished sense of what other people are. It’s no wonder that people like the self-styled philosopher-kings at Anthropic are so seduced by their creation.
Whether democracy has ever really existed – whether it could possibly exist under the capitalist mode of production – is, for many of us, an open question. But anyone acting in good faith would have to agree that whatever AI companies are doing isn’t democratic in the slightest.
The Genuine Intelligence Project is an initiative from the North Carolina branch of the American Association of University Professors. Check out our website, and follow us on Instagram and BlueSky!