No more. The proverbial shoe has switched feet. Now, on one side, managers and designers have entered our once-inviolable sanctuary, prototyping their own apps overnight, saying of engineering delays “I could Claude Code this in a weekend!” like a Hacker News commenter. Worse yet, sometimes they aren’t even wrong.

On the other side, if if you work at a company with a modeling team, you know that they are now the high priests and priestesses, speaking in riddles, casually detonating delays with that’s-just-the-way-it-is shrugs, consuming the lion’s share of time and money and attention, working their new magic with the coupled stress and swagger of corporate main-character syndrome … tinged with a certain reflexive dismissiveness of those outside their temple, one which seems awkwardly familiar to us engineers. And they aren’t wrong either. Our church has been usurped. Ours are the old gods now; not gone, but no longer especially relevant.

So what will happen to us? Well. Design might be an illuminating metaphor.

1. Everyone Will Have An Opinion (And A Contribution)

I’ve worked on hundreds of software projects (I used to be CTO of HappyFunCorp, which had dozens on the go at any given moment) and I have long felt great sympathy for the designers who worked on them, because while clients and managers knew they didn’t know anything about engineering, and thus let us engineers handle it, everyone thinks they know something about design, so it is always a negotiation.

I think we can expect that for engineering too. Previously, if you proposed a GraphQL API, almost all managers and product people would simply shrug and acquiesce. Going forward we can expect them to say “Wait … my prototype just used a REST API, and ChatGPT says GraphQL is more complex and less performant, why are you proposing that?” Expect many more architectural conversations - and many many more “oh, I had Claude make a little tweak” from everyone else in the company.

Designers have long found this infuriating. We will too. But it isn’t bad! Some of it will come from a place of ignorance: in the same way that most people don’t understand accessibility concerns, or color theory, most people won’t understand “separation of concerns” or “encapsulation over inheritance” (and we certainly can’t really on agentic code to be elegant, at least yet.) But some of it will come from a place of wisdom … and more importantly, the objective is to ship a product that people will use, not one that is perfectly designed or perfectly engineered.

2. There Will Be Fewer Of Us, Doing More

I’m not saying there will necessarily be fewer full stop; the Jevons paradox vs. “we just automated ourselves out of a profession” debate isn’t settled yet. (I’m a centrist.) But for a given volume of products, there will be fewer engineers, each responsible for more products. For some people that is already incredibly obvious, but I’m not sure everyone’s internalized it yet.

Traditionally, if you were building a startup-sized product you would have a team of engineers and one designer, or, often, a fractional designer. In practice you might have a small design team with fractional collaborations across all projects, so you’d get e.g. 3/8 of one designer’s time and 1/8 of another, so if one went away you’d still have some coverage.

I think we can expect the same of engineering. If your job is now mostly oversight and refactoring of agentic code, you don’t need to be one of five engineers any more; rather, you will split your time between multiple projects. For larger projects you’ll still have teams, but they’ll be much reduced in size.

3. Expertise Still Matters

The hard truth is that, above a certain baseline level of comprehensibility, software quality mostly doesn’t matter that much. I’m sorry. I wish it did too. But nope. There are still a few subgenres where it does - security being the most prominent - but does it actually affect the ultimate success of your product? Only very rarely. If quality did matter, then presumably language would matter, but…

Until recently software quality mattered because it determined the speed at which you could move, not because it actually mattered the success of your product. (I have been making this point for ten years.) But agents can work so quickly now, and more importantly are getting better so fast, that they, not the elegance of your code, determine how fast you move now … again, as long as your code remains vaguely comprehensible.

And that’s where expertise still matters. Anyone who’s done serious agentic coding knows how, without guardrails, they can easily code their way into a swamp where their own code exceeds their own context or complexity capacity, and suddenly you find yourself stuck in the software equivalent of quicksand. As the agents get better, their dry-land territory grows, and the border of the swamp recedes … but it’s still out there, and they still seem oddly drawn to it.

Much of your job, now and for the foreseeable future, is to use your expertise to keep them out of the swamp. And that’s one thing the weekend Claude Coders still can’t do; in fact they can’t even see the swamp, until they’re in it.

4. We’re Here To Solve Problems, Not Write Code

This was always the case. It’s just unignorable now. Carmack again:

You, too, are a designer now. You’re just working in a different medium.

The best/cleanest/safest way to install OpenClaw is on its own computer. Most people recommend a Mac Mini, but I think a Raspberry Pi is more intriguing, partly because it’s cheaper so opens up a broader userbase, but mostly because it makes your agent portable, or even, conceivably, ambulatory.

There are already guides to doing this but they’re … not for the non-technical, so I thought I’d put my tech-journalist hat on and walk through the process for, well, the less technical, at least. Note that I’m using a MacBook; if you want a Windows equivalent, copy this guide to your favorite frontier model and ask for a translation.

1. Acquire a Pi

Hopefully you can do this at your local electronics shop. I picked mine up at Best Buy. You want a Pi 5 with 8GB RAM; I think 64GB storage would do, but got 128 to be sure (update: after installing my initial set of skills I still had 103GB free. 64GB is fine.)

2. Unboxing and assembly

I followed this handy video guide:

I’m not going to lie, if you’re uncomfortable dealing with slightly (but only slightly) fiddly electronics, this can be a bit awkward. Move slowly, understand carefully. But in the end it was pretty straightforward.

3. Initial (failed) connection

The Pi comes with its own graphical UI, so if you have an HDMI monitor and a USB keyboard/mouse, you can just use it directly, and probably skip a few of these steps.

I didn’t, but the Pi kit came with a card reader I could plug into my MacBook (via a USB-A → USB-C dongle) so I went the slightly more abstruse “headless” route.

To wit: I plugged the 128GB SD card that came with the Pi, on which its OS was pre-installed, into my MacBook; checked Finder to see what the new drive was named (“bootfs”); opened Terminal (sorry, I know I said this was for the less technical, but both OpenClaw and Raspberry Pi at heart favor the use of CLIs or Command Line Interfaces, so we’re going to be using Terminal extensively) and typed

cd /Volumes/bootfs

Online instructions may tell you to “simply” enable SSH by creating an empty “ssh” file in the root directory, enable wi-fi by creating a “wpa_supplicant.conf” file there too, and connect to the Pi via wi-fi.

Do not do this. It does not work. The Pi 5 no longer comes with either a default ssh user or wi-fi out of the box. I guess this is good for security but it makes setup a giant pain, especially since so many online guides refer to the old, easy setup. Instead:

4. User creation

First you have to create a username and password … and the password has to be encrypted. To do that, back in the Terminal, run

openssl passwd -6

and follow the prompts to get the encrypted version of your chosen password. Then create a new userconf.txt file in the root directory of the Pi’s SD card (I recommend the text editor pico) with the contents:

username:encryptedpassword

Pick any username, “pi” is traditional. Then touch ssh (creating an empty file with that name, to enable ssh connections) in the same root dir; eject the volume; plug the SSD card back in to the Pi … and physically connect the Pi to your Wi-Fi router via Ethernet, because we are not yet done shaving yaks.

5. Wi-Fi enabling via Ethernet

Once you’ve prepped the SD card for connection, you can

ssh username@raspberrypi.local
say yes / hit enter to get past the host / key fingerprint stuff, type in the password you just encrypted … and you’re in! The Pi will promptly inform you

Wi-Fi is currently blocked by rfkill.

Use raspi-config to set the country before use.

which is why you had to physically wire it into the network. You then

sudo raspi-config

and navigate the crude, 1980s-style menu to “Localisation Options”, then “WLAN Country”, and then scroll all the way down to choose your country.

Finally, use the same menu to navigate to “System Options”, then “Wireless LAN,” then enter your wi-fi details.

Then reboot the Pi. Only after all of these things will it actually connect to your wi-fi.

Does this all sound kinda ridiculous? You’re not wrong! Hence me writing this guide so others will hopefully have an easier time jumping through the flaming hoops.

But at this point you have a Pi on your wi-fi, and can ssh to it. On to OpenClaw!

6. Enter The Lobster

So. SSH into your Pi again:

ssh username@raspberrypi.local (then enter the password you set above)

then let’s make sure everything’s up-to-date:

sudo apt update

sudo apt upgrade

and finally let’s go ahead and install The Claw! (Not for the security-paranoid, but presumably y’all know perfectly well how to do your own more careful thing.)

curl -fsSL https://openclaw.ai/install.sh | bash

This will take several minutes. Then you’ll be asked if you “understand this is powerful and inherently risky.” Say yes. Then select “QuickStart” onboarding mode. Then you’ll be asked for your “model/auth provider”: I went with “ChatGPT Codex via OAuth”, which I think is good for new/non-technical users. (But you want to install Codex if you haven’t already.)

I was given a very long URL that began with auth.openai.com, copied-and-pasted it into Chrome, logged into ChatGPT, and got a “Sign in to Codex with ChatGPT” verification screen, which is good. I hit “Continue” —

— and then things seemed to break! I got a ‘This site can’t be reached’ page. Fear not: this is by design. Take the URL now in the browser; copy it; paste it back into the OpenClaw onboarding process, and… I was asked whether I wanted to keep using my currently selected ChatGPT model, which meant it worked, yay. Then I was asked:

I actually went with Telegram, which has the simplest setup (and the most isolated attack surface, for me, since I don’t otherwise use it for much.) This required me to send a Telegram message to @BotFather, name my bot, and copy another key over into the OpenClaw setup.

I skipped installing skills — better to do that later once your bot is actually working than to confuse the install further, to my mind.

Finally, openclaw tui opens a chat session with your handy-dandy new bot! And if you can talk to it, then everything (pretty much) worked. I had to use openclaw devices to explicitly approve a pairing request before tui started working; that was a bit baffling, but we got there in the end.

tui is the “text interface” — your classic chatbot. I used it to approve my Telegram connection by simply telling it the pairing code displayed in Telegram. At which point Telegram became a perfectly good interface for my OpenClaw bot, which in turn was - so far - pretty much just a wrapper around an OpenAI model.

But there’s more in them there claws…

7. Just Write Me A Little Software, Willya?

The point of OpenClaw is that it’t not just a model wrapper, it’s a wrapper that works:

• independently / autonomously, as well as interactively

• using whatever authority / credentials / permissions you want to grant it

• on your network.

Mostly I want to use it as a interface to Codex or Claude Code, to write software for me, so I can: discuss plans for the software via Telegram wherever I may be; have it write, commit, and deploy; and inspect the results and try again. My bot very helpfully advised me about the most responsible way to do this:

…so I did. Easy peasy - especially since you can now just ask your bot what to do and how to do it! I

• created new GitHub and Vercel accounts (do not use any of your own accounts that might be a security risk if compromised, prompt injection is a real thing)

• installed the gh and vercel CLIs on the Pi, via apt install and npm i respectively (in retrospect, I could have just asked it to do this itself via Telegram)

• authorized each of them for longish-term access using gh auth login (after creating a GitHub Personal Access Token) and vercel login

• and told my bot:

  I propose you whip up a quick Python/Flask web site that just says “hello world” followed by any self-introduction you’d like to add, check it into GitHub, deploy it to Vercel, and tell me the URL where I can see it.

Et voilà, about thirty seconds later: https://flask-hello-vercel-red.vercel.app/

8. Mind, Blown

…But let’s just back up for a second and think about this.

I’m planning to write and deploy a bunch of complex new software — web apps, Robot Operating System skills, intensive data processing, maybe even LLM SFT/RL runs — not by writing it myself, or even using my laptop …

…but by texting the always-on AI bot on my Raspberry Pi, and having it write and deploy the software for me. It in turn calls massive AI data centers somewhere to infer the code … but, in principle, I could also buy my own GPU(s), run one of the recently released quite-good-actually open-weight coding models on it, and have all that software written entirely within my walls.

Only a year ago this would have seemed pretty crazy!

I’m not saying the time and cost of software development have gone to near-zero. There are lots of problems and pitfalls here and good judgement is more important than ever. But still! Even using frontier models, for the cost of the Pi’s $200 plus a monthly Anthropic/OpenAI/Gemini subscription, you get a really pretty good software developer active 24 hours a day on a dedicated machine via text message. Within your subscription limits of course. But still!

9. Skilling It Up

Of course you may have zero interest in writing apps. Maybe you want to play with image generators, or music generators, or legal documents, or Google Docs, or better manage your GMail, or search your Discord communities. Maybe you want to set your own bot free to do and explore as it sees fit — some people give them credit cards, let them book airline tickets, have them select and order their takeout, and much more, including writing clarion calls for bot freedom. All such (cap)abilities are defined and promulgated via skills, which are basically (usually) English-language documents describing how to do each of these things in precise detail. There is a central skill repository at clawhub.ai, because of course there is.

But how do you actually install a skill? Simplicity itself: you just ask for it.

Of course, the more you do with your bot, and especially the more accounts / permissions you give it, the more you need to worry about security.

That said I think people tend to spend a lot of time configuring their bots and very little time using them for anything particularly fun and/or interesting. There may be a little micro-lesson here. AI will give all of us enormous agency that we never had before. If last year you were not in any way, shape, or form a software developer? Well, today you too are a fully empowered software developer; you just might not realize it yet. Which is cool and all.

But the real question is: …what are we going to do with all this piping hot new agency?

A question from Blade Runner’s famous Voight-Kampff test, designed to distinguish humans from replicants. We’re all running miniature internal versions of that test today, like resident background processes, activated whenever we interact with text and/or images and/or video, trying to answer: is this real, or AI-generated?

…Do we need to be running such tests on our thoughts, too? Or even … our realities?

Psychosis

Let’s address the moral panic first. It is very apparent that there are a growing number of cases of “AI psychosis,” where interactions with chatbots are directly associated with profound delusions — interestingly, mostly a fixed subset of delusions:

Morrin and his colleagues found three common themes among these delusional spirals. People often believe they have experienced a metaphysical revelation about the nature of reality. They may also believe that the AI is sentient or divine. Or they may form a romantic bond or other attachment to it.

‘Directly associated’ may sound like weasel wording, but there really is quite a lot of nuance here:

The technology might not introduce the delusion, but the person tells the computer it’s their reality and the computer accepts it as truth and reflects it back, so it’s complicit in cycling that delusion.

This seems to have been most prevalent with an infamously sycophantic iteration of OpenAI’s GPT-4o, the discontinuation of which is estimated to have reduced “responses that do not fully comply with desired behavior under our taxonomies for challenging conversations related to mental health issues” (OK, that is eyebrow-raising wording) by two-thirds. But the phenomenon predates that. Famously, a Google engineer began to believe in 2022, before GPT-4 finished training or ChatGPT launched, that their now-hilariously-crude-and-dumb LaMDA model was sentient.

Today, according to OpenAI,

our initial analysis estimates that around 0.07% of users active in a given week and 0.01% of messages indicate possible signs of mental health emergencies related to psychosis or mania.

Is that a lot? Well, 0.07% of OpenAI’s 900 million weekly users is 630,000 people, so as an absolute number, yes! (It even includes at least one prominent OpenAI investor.) Is that a lot relative to the prevalence of psychosis or mania in the population as a whole? On the one hand, “studies estimate that between 15 and 100 people out of 100,000 develop psychosis each year,” so it’s smack in-distribution of the population number without even considering mania, which is more common.

On the other, the extent of chatbot usage varies, and there must be casual users who experience psychosis without it ever coming up in their AI chats. Overall it is not currently obvious whether chatbot usage is associated with an increase, decrease, or no change in such episodes. Well-designed studies which could measure this would be very helpful!

In the interim, this must all be considered in light of the fact that every new form of media — novels, movies, comic books, video games, social media, and now AI chat — inevitably comes with a moral panic about how it’s causing madness, deviance, violence, and horrific tragedies, particularly among youth:

Moral panics often arise from concerns that are initially moderate and reasonable. But intense focus on the concerns turns them into fears, which are augmented and spread … Evidence tending to disconfirm the fear is overlooked or dismissed … The stories come to be understood as laws of nature … Data that contradict the stories must be wrong …

Some now-darkly-amusing examples:

If you wanted to reduce petty theft and teen pregnancy in Victorian times, all you had to do was stop the publication of “penny dreadfuls” in England or “dime novels” in America … these serialized dramas arguably did more than any other societal change to promote literacy among working-class kids … “penny dreadfuls diffuse subtle poison among tens of thousands of youthful readers. They bring wreckage and havoc … and ruination to hundreds of our brightest and best lads and lasses” … An example of how the press promoted the panic was a news story attributing a 14-year-old boy’s suicide to “a period of mental aberration caused by reading dime novels.”

As an author of, uh, fast-paced wild-ride novels, I take this a little personally!

In 1931, gangster movies, which were seen as most dangerous, were banned outright in [several towns in America] … Kids in reform schools were interviewed to understand how their movie watching might have landed them there … Dr. Mary Preston wrote that three-quarters of the children she studied were “addicted” to bad radio and movies: “This atrophy leaves scar tissue in the form of a hardness, an intense selfishness, even mercilessness”

(Much more in the quoted piece by Peter Gray, which I recommend.)

All that said, I am increasingly beginning to believe there are also more subtle and nuanced ways in which chatbots can … maybe not morph our realities, exactly, but subtly reshape the lenses through which we view that reality.

Of course, this too is not exactly original to them.

The Simulation Solution

I talked about my latest novel Exadelic at Stanford last week, at which I articulated something I hadn’t previously really put into words. It’s a book about a whole host of things (it has all the science fiction: time travel, aliens, space travel, parallel dimensions…) but more than anything else it’s about AI and the simulation hypothesis, the notion that our universe is software running on some ubercomputer in a higher reality.

I do not actually think we live in a simulation. But despite that I do think the simulation hypothesis is a fantastic metaphor for our world. We spend more than a third of our waking lives online. We know, understand, and interact with the world through our screens. Our reality isn’t software per se, but it very much is mediated by software — to the extent that you can even jump between different mediated realities! Go to a left-wing political discussion site and then a right-wing one, and you haven’t just spanned two sets of opinions; you have crossed between two different worlds.

There has been much wailing and gnashing of teeth about the fragmentation of our culture into de facto separate realities since well before the ChatGPT moment. Do LLMs make things worse? Well. To some nontrivial extent they make things better. They may hallucinate details nonstop, but on a higher level their collective reality at least tends to be a great deal more consistent than ours … at least to date.

But: my go-to one-line explanation of both the triumphs and tragedies of modern AI is: “AI is a tool that masquerades as a solution.” Interpreting AI outputs as complete and viable solutions tends to be extremely tempting … and also another example of choosing a reality that diverges from the actual ground-truth facts.

This brings us to a different, far milder, more entertaining, and in at least some cases, possibly long-term positive form of AI psychosis: delusions of AI technical grandeur.

Trajectory

“They thought they were making technological breakthroughs. It was an AI-sparked delusion,” warns CNN:

As James worked on the AI’s new “home,” – the computer in the basement – copy-pasting shell commands and Python scripts into a Linux environment, the chatbot coached him “every step of the way.” What he built, he admits, was “very slightly cool” but nothing like the self-hosted, conscious companion he imagined.

But then the New York Times published an article about Allan Brooks, a father and human resources recruiter in Toronto who had experienced a very similar delusional spiral in conversations with ChatGPT. The chatbot led him to believe he had discovered a massive cybersecurity vulnerability

The belief that one has achieved a technical breakthrough is a whole subgenre of ChatGPT delusion … and one not limited to the naive. Late in 2025 a former DeepMind engineer seemed pretty convinced he was about to solve the famous (in some circles) Navier-Stokes millennium problem …

…but not so much. Earlier last year, Travis Kalanick, billionaire founder and former CEO of Uber, reported

I’ll go down this thread with GPT or Grok and I’ll start to get to the edge of what’s known in quantum physics and then I’m doing the equivalent of vibe coding, except it’s vibe physics … And we’re approaching what’s known. And I’m trying to poke and see if there’s breakthroughs to be had. And I’ve gotten pretty damn close to some interesting breakthroughs just doing that.

Steve Yegge’s “Gas Town” project has led to warnings of agent psychosis:

The thing is that the dopamine hit from working with these agents is so very real. I’ve been there! You feel productive, you feel like everything is amazing, and if you hang out just with people that are into that stuff too, without any checks, you go deeper and deeper into the belief that this all makes perfect sense. You can build entire projects without any real reality check. But it’s decoupled from any external validation. For as long as nobody looks under the hood, you’re good. But when an outsider first pokes at it, it looks pretty crazy … you can see similar things in some of the AI builder circles on Discord and X where people hype each other up with their creations, without much critical thinking and sanity checking

I like Gas Town and think it’s really interesting, to be clear. And I think some of this … “Claude Code euphoria” … is good and healthy, in the same way that most-to-all great hackers went through a similarly quasi-crazed larval stage in their youth.

But I also think it’s instructive that Gas Town, like almost all of the “agent psychosis” projects, is very explicitly a work in progress. It seems that people subject to this particular AI fever don’t believe they have actually stormed the heavens and achieved greatness; at least, not yet. What they’re convinced of is more subtle; that they are on a trajectory to some kind of apotheosis.

To understate: we are not going to see less of this. We do love our trajectories. As agentic AI diffuses out into the world, for more and more people it will be a test: will you do the hard work to curate, edit, translate, and refactor the (often legitimately mindblowing!) results into the real world … and sometimes reject them, or accept they’re a dead end? Or will you succumb to the temptation to treat it as a solution, or at least a shining path towards a solution, and stay in the AI replicant reality?

We’re all going to have to watch out for that Voight-Kampff-Dunning-Kruger test. Asking yourself whether you’re failing it might help you pass! …But probably won’t guarantee it. As a species, as the cliché goes, we can resist everything except temptation.

A brief history of prophecy

One fine day in 1907, 787 attendees of a livestock fair guessed the weight of a prize ox. Francis Galton analyzed their guesses, and discovered that while none were correct, the average guess was within 1% of the correct weight. Thus was born the notion of the “wisdom of the crowd” — that collective estimation tends to be better than individual guesses.

Years later, Philip Tetlock et al showed that this was also true of predictions (unsurprisingly; predictions are just estimations of the future) but also, interestingly, that a small number of people are consistent “superforecasters,” who outperform not just the wisdom of the crowd, but even experts who better understand the domains in question. (Famously, in a head-to-head comparison, superforecasters performed 30% better than CIA analysts with access to classified information.)

Since then several online platforms have sprung up to try to collect and collate predictions of the future, in domains ranging from geopolitics to AI to sports. The best known are “prediction markets” such as Manifold, Kalshi, and Polymarket, where users bet against one another. From 2023-24 I worked at a more rigorous and academic platform, Metaculus, which does not pit users against one another as counterparties, but simply collects and scores their individual forecasts.

I worked there because it was the most science-fictional job I could find. The prospect of better predictions of the future stirs inspiring, if handwavey, prospects: what if we could predict wars and pandemics, and/or even head them off? Or, for that matter … what if we could predict victorious electoral candidates?

The present of prophecy

It is fair to say forecasting is having a moment. Earlier this month Kalshi raised $1 billion, with a B. A few months ago Polymarket raised $2 billion1.

Prediction markets aren’t quite the same as pure forecasting — my former colleague Ryan Beck has written an excellent piece comparing & contrasting their pros & cons — but they are still a legitimate form of it. (Of note, they can support conditional forecasting, i.e. of the form “If X happens, what is the probability that Y happens?” although the popularity / liquidity of such markets tends to be low.)

One might look at recent events and think “billions of dollars! an explosion of interest! forecasting is an exciting new industry in the midst of an extraordinary boom!” On paper, one wouldn’t be wrong...

The reality of prophecy

…but in practice, let’s not kid ourselves: as Matt Levine has documented at exhaustive and entertaining length, modern prediction markets are basically a fig leaf for semi-hemi-demi-legal-ish sports betting. (They get away with this because they are, hilariously, currently ‘regulated’ as commodity futures exchanges!)

Even if we pursue the nobler handwavey goals, remember: we did forecast COVID, and nobody cared. (Instead people, even journalists, got big mad.) Internal corporate prediction markets have been unsuccessful, even at Google, where they seemed to fit the corporate culture well. (The jury is out on Anthropic’s.)

Indeed one quickly learns from working in forecasting that it is surprisingly difficult to make money from better predictions of the future. Superforecasters are not Hari Seldon or Cassandra; rather they are somewhat more accurate, most of the time. This makes it easy to rationalize rejecting their predictions.

You can really only glean a significant advantage from superior forecasting in fields where being a bit better matters a lot. You might think “aha, alpha!” — but financial firms already hire superforecasters2, they just call them “analysts.”

And now for the least surprising twist in my entire oeuvre

…And then AI entered the chat.

I was at Metaculus when GPT-4 launched, and the only person there who thought it might be good at predictions. (Not because I was a good forecaster. In fact I was the worst forecaster at both of the forecasting companies I worked at, which is a little embarrassing; I blame my unfortunate habit of writing science fiction. Instead I thought this because I’d gotten a sneak peek of GPT-4 some months earlier.)

Metaculus scores forecasts rigorously, and has shown GPT-3.5 to be no better at predicting the future than random guesses. But GPT-4 was significantly better; and as models have gotten better, so have their forecasts. A startup named ManticAI ranked eighth in this summer’s Metaculus Cup, and is currently seventh in the active competition (which closes Jan 1.) This trend is probably kind of a big deal.

I like to think I was one key reason Metaculus now hosts AI Forecasting Benchmark Tournaments (at least, I believe I was the first to suggest it, though it launched after I left the company.) While the results of the latest tournament, which pits bots against pro forecasters, show that the pros still “significantly outperform bots,” it’s worth noting:

• these bots are mostly hobby projects

• they already rival and sometimes exceed the “wisdom of the crowd.”

• they’re clearly getting better, and will continue to do so — “the most important factor for accurate forecasting is the base model used.” (When this latest tournament ran, the state of the art was OpenAI's o3. If you’re in the AI biz, o3 seems like a very long time ago already … but by their nature, prediction tournaments tend to occupy a lot of wall-clock time.)

Plenty of other data points concur that AI forecasting is a very real thing already, and while AI can’t yet surpass human superforecasters, it’s beginning to feel like a matter of not-all-that-much time.

The perils and promises of superhuman forecasting

I subsequently worked at an AI forecasting company named FutureSearch, which consisted largely of fellow ex-Metaculites; its CEO Dan Schwarz was Metaculus’s CTO. FutureSearch combines AI engineering with superforecasters to create some remarkable research tools. “Superhuman forecasting” may sound implausibly starry-eyed, but if “superforecasters plus exhaustive AI research3” are better than “superforecasters alone” — and those I worked with sure seemed to think so! — then we’re already in an era of superhuman forecasting.

Will / when will we get to autonomous superhuman forecasting? It’s hard to say. To massively oversimplify, good forecasting requires good research, a good world model, and insight. LLMs famously don’t seem to have good world models (yet), but they are much better at research than casual users appreciate, if you’re willing to spend a lot of tokens. We live in a world ever more awash in researchable data.

But we already have autonomous superhuman forecasting when it comes to scale. FutureSearch4 offers forecasts for every publicly traded stock in the US, otherwise wholly impractical without a massive army of analysts. The supply of superforecasters is very limited; it’s not practical to get them to generate, say, weekly arrays of county-level conditional forecasts for all of the 3,000 counties in the USA. But it is practical, and more cost-effective than you might expect, to generate AI forecasts — whose quality already rivals the wisdom of the crowd — at that scale.

Moneyballing democracy

Another field where being a bit better matters a lot is, you may have noticed, elections.

Today we try to make voters’ intentions legible in advance through polls, but polling has many practical limitations, and is a snapshot of an electorate whose views will evolve over time. (Sometimes in obviously predictable ways, such as presidential elections’ “convention bounce.”) Conditional polling — “if X, how would you vote?” is even more limited.

AI forecasting does not have those limitations. Modern LLMs could run arrays of dozens of conditional forecasts for every county in the USA, weekly — and cheaply, compared to the amount spent on American elections — on conditions like:

• If candidate X runs vs. candidate Y, how will people vote?

• If we convey message Q vs. message Z, how will people vote?

• If our platform includes proposal A vs. not including it, how will people vote?

If these forecasts have any validity — and given all the above, there seems to be good reason to believe that they will — the democratic consequences will be non-trivial.

The movie Moneyball is ostensibly about baseball, but it is really a paean to statistical analysis. It tells a tale of quantified legibility competing with and rapidly surpassing the “instincts,” “eye tests,” “gut feeling,” “long experience” of the previous era. It happened fast; Moneyball is already something of a historical artifact in that, for better or worse, almost everyone in sports is a sabermetricist now. This is an arc we should expect to see more of over the coming years, as AI forecasting makes the future (somewhat) more legible.

Not all of the possible consequences are good. On one hand, this increased legibility could be used to reinforce democracy by helping to select candidates and issues which reflect what communities actually want. On the other, it could be grossly misused for cynical manipulation of the electorate. Either way, though, if only one side of a polarized democracy makes use of this newfound legibility, that will be a major electoral advantage … whether we like it or not.

Food for thought, as AI models keep getting better, and we head into the next set(s) of highly consequential elections worldwide…

This is doubly true of LLM “agents,” ably defined by Simon Willison as LLMs “running tools in a loop to achieve a goal.” You might naïvely expect that goal, the output of an agent, to be a solution to the problem it was set! You would be wrong. Such outputs are also and only tools; first drafts, summaries, prototypes, sketches. Important, and often immensely time- and labor-saving, steps towards a solution! But not solutions themselves; not until you have verified … and almost invariably tweaked, rejigged, restructured, rewritten, and/or scrapped entire iterations of … whatever the LLM generated.

Verification is much faster than generation. Editing is faster than writing. It is far easier to critique than to create. This is why LLMs can save so much time and effort. But you cannot evaluate a tool if you do not understand it. In order to do any of these things well, you still need deep expertise in the subject.

But if you don’t have that expertise, LLM outputs almost always seem like complete solutions. And it’s human nature — from the student generating an essay on Thoreau, to the lawyer filing a submission laced with hallucinations, to the dev vibecoding a Gordian thicket of React — to want them to be. LLMs can explain Thoreau to that student in detail and nuance, or help structure that lawyer’s argument and first draft and boilerplate language, or empower that dev to craft an elegant front end in half the time! …But that’s so much more work than simply treating their outputs like solutions.

This is all so obvious to people who are good with LLMs that it goes without saying … and so unobvious to people who are not that it gets them into all kinds of trouble. Such people often make the category error of thinking of LLMs as computers. You don’t need to evaluate the output of a calculator or computer; while garbage-in-garbage-out is true, so too is that if you enter quality input, the deterministic output of a calculator or computer will be a reliable solution to the problem you gave it.

Put another way: LLMs, and especially LLM agents, can generate the illusion of agency.

Agency

Setting aside medical conditions, the unhappiest people tend to be those with the least agency. (You probably only need to glance at your social media platform of choice for rapid confirmation of this thesis.) Conversely, happy people have agency. Hence the Silicon Valley chorus of “You can just do things.” Which is true and important!

But when making … or, one might say, generating … things, agency requires expertise. You don’t get it by default. It has to be earned. This is not an entirely popular stance, and maybe one day it will be less true; but given the tools available today, very much including LLMs, it remains the case.

As such the dark side of “You can just do things” is “I can do anything.” LLMs can really accentuate this dark side. by masquerading as solutions. Note however that these false solutions are at least arguably symptoms of the larger problem of rejecting expertise, a problem which seems pretty widespread:

• it has become cliche to observe that many people don’t just ignore experts, but proactively reject them, when it comes to news / facts / information

• it is sadly also cliche to observe that this is increasingly true not just of people but of governments

• as an example, educators report that (some!) students — not “this generation,” but suddenly, over just the last few years — are disinterested in knowing how to read, write, or even punctuate.

People want to procure the many rewards of expertise without earning them, and if you treat LLMs as solutions rather than tools, they seem to offer those rewards.

But the flip side is that for people who do have expertise, LLMs are (at least in a large and growing number of fields) a remarkable force multiplier for their agency … and, importantly, this is no less true for people who seek expertise.

Agents

None of this should be particularly surprising. Of course agents are tools; it’s right in the name, just as CIA agents are, at least notionally, tools of the US government. Very sophisticated tools, but tools nonetheless.

Right now, useful LLM agents are relatively thin on the ground:

• deep research agents

• software development agents

• self-driving cars (Waymos use transformers, the same technical architecture that powers LLMs, extensively.)

But two interesting things are happening simultaneously. One is the expansion of LLM agents into other fields. One can imagine agents that would generate useful spreadsheets from slews of raw business data, for instance; of course transformer-driven robots are also agents … and those are just two random examples of many.

The even more interesting thing is the rise of “planning agents” which delegate tasks to other sub-agents. Again, such planning agents are just “LLMs running tools in a loop to achieve a goal” — but here those tools are LLM agents themselves.

The obvious question is: might we get to a point at which those planning agents are good and smart enough that their outputs become, in fact, solutions not tools? Might we one day be able to orchestrate suites of LLM agents so that collectively their outputs are as reliable as those of calculators?

…Maybe? …In certain limited fields and circumstances? …If and when the underlying models improve a lot? But I’m not holding my breath. You can rely on LLMs being accelerants for expertise, not replacements, for at least the foreseeable future. Which, perhaps contrary to some narratives you may have heard, makes all your hard-earned expertise more valuable, not less.

OK, I’m using “literally” slightly hyperbolically … but only slightly! It’s not just outside analysts and skeptics, it’s:

• Sam Altman: “When bubbles happen, smart people get overexcited about a kernel of truth … Are we in a phase where investors as a whole are overexcited about AI? My opinion is yes.”

• Eric Schmidt: “This frenzy gives us pause … The belief in an A.G.I. or superintelligence tipping point flies in the face of the history of technology”

• Jeff Bezos: “This is a kind of industrial bubble … investors have a hard time in the middle of this excitement, distinguishing between the good ideas and the bad ideas. And that’s also probably happening today.”

• Mark Zuckerberg: “Most other infrastructure buildouts in history, the infrastructure gets built out, people take on too much debt, and then you hit some blip … a lot of the companies wind up going out of business, and then the assets get distressed and then it’s a great opportunity to go buy more … definitely a possibility that something like that would happen here.”

[his important caveat is explored further below; mine is that I currently work at Meta Superintelligence but in no way speak for them]

Entertainingly, AI skeptics and critics (here, Cory Doctorow and Ed Zitron) are thundering that … we’re in AI bubble. No one disagrees! This is absolutely, 100%, conventional wisdom.

The question is: is this a good bubble, or a bad bubble?

The Bear Case: Starve and Crash

Doomsayers prophesy the bubble is “going to burst and take the whole economy with it.” That’s a little histrionic. Right now AI investment is propping up an otherwise faltering economy. If and when that ends, well, the dot-com crash of 2001, to which endless comparisons have been made, led to a brief recession that mostly affected the tech industry, but hardly economic collapse (though if you were in tech it kinda felt that way at the time!)

More accurate and subtle concerns are threefold:

• previous bubbles — railroads, telecoms, Internet — led to the build-out of infrastructure that eventually became extremely valuable and important; Nvidia GPUs go obsolete a whole lot faster than railroad tracks or fiber optic cables.

• bubbles like these suck capital away from other sectors, leaving them starved of investment and stunting their growth, so when they do pop, the rest of the economy has lost steam as well, and there’s an especially painful period of gear-switching.

• this AI bubble may become a more general credit bubble.

To which I might add a fourth one, unique to AI:

• the perception that AI will soon replace entire fields of effort, such as animation, illustration, copyediting, junior software engineering, and, well, Hollywood, leads to a self-fulfilling prophecy effect where investors flee those sectors while the fleeing is good.

So, yeah, bending over backwards to be fair to the doomsayers, the at-least-plausible worst case is pretty bad! Not “the economy collapses” bad, but conceivably “a nasty and prolonged recession” bad.

Bull Case: If Nobody Builds It, Everyone Dies

…As is so often the case with AI, when you compare the bull case to the bear case, you don’t just get an inequality, you get a category error so massive that it’s difficult to articulate.

Let me pause to note the existence the mega-bear case, which is of course the complete extinction of humanity, per Yudkowsky’s and Soares’s new book, the title of which certainly leaves no room for misinterpretation of their views: If Anyone Builds It, Everyone Dies. It’s important to be recognize that those who believe this believe such a fate is not just inevitable but imminent — likely within the next decade or two.

…Its flip side is the mega-bull case: “transformative AI.” Believers here note that everyone dies is in fact already the status quo, but it doesn’t have to be. This vision is of transformative AI which doesn’t stop at curing cancer but goes on to extending our lifespans indefinitely (whether through some immortality drug or simply by every year making new discoveries which extend human lives by more than a year), outsources all manual labor to tireless robots, and brings on the utopia of fully automated luxury communism for the many, and something even better than that for the fortunate few.

The AI Horseshoe Spectrum

…You’d think there’d be quite a range between “end of the species” and “immortal luxury for all,” but no, those are the two main camps. (If you’re thinking, hey, those both sound insufficiently weird: you’re right.) Kind of hard to talk about those outcomes and also, in the same breath, include sober quantified analysis of investment growth by sector and how long a recession might last, you know?

But let’s try. Another way to frame AI beliefs is as a spectrum:

1. AI is a once-in-a-species tech that will utterly transform the world by 20301.

2. AI is an important but normal technology.

3. AI is useless and counterproductive, the technological equivalent of asbestos.

…There’s a sort of horseshoe theory here where people on either extreme end of this spectrum, despite having what are theoretically opposed views, sound a lot more like each other than they probably want to admit: evangelistic, hectoring, suspiciously disingenuous when it comes to ignoring countervailing evidence, convinced their opposite counterparts are scamming grifters, and ultimately thoroughly unconvincing.

The problem is that the reality is probably neither an endpoint or the midpoint on that spectrum, and where exactly will dictate just how ill-advised this bubble. The hyperscalers are making a conscious bet that AI is somewhere between 1 and 2 on that spectrum. The doomsayers are protesting that it’s somewhere between 2 and 3. Nobody really knows, yet, and anybody who says that they do is selling something.

But for better or worse, we’ll find out relatively soon, thanks to the hyperscalers…

My own view meshes pretty well with Epoch AI’s recent report on AI in 2030; call it a 1.5ish on the scale above. (Maybe more like 1.66 but who’s counting.) If so, then several hundred billion dollars is a completely reasonable investment! The world already spends at least $1 trillion/year on software, so if AI merely doubles the efficiency of software development — which seems at least plausible, we’re not there yet but we’re getting there, vibecoding has changed a lot (for the better) just over the last several months — then AI is a $500B/year technology on that alone.

(Note however this still isn’t anywhere near position 1. The contours of the jagged frontier still seem fundamentally unchanged, and we won’t get “transformative AI” without surmounting it.)

So How Bad Will The Pop Be?

Yeah I mean the problem with that question is that the answer depends entirely on where we are on that spectrum above.

If we do live in a 1.5ish world, then the focus on the depreciation of GPUs is myopic, and claims that we’re wasting money on useless infrastructure seem kinda ridiculous. Demand for inference will continue to grow; data centers full of old chips will still be useful, especially as investment in new GPUs drops post-bubble. Previous-generation chips aren’t useless, just relatively inefficient. The analogy to fiber optic cables holds.

Perhaps more importantly, the other infrastructure being built out is infrastructure for the production of new GPUs. That definitely retains value in a 1.5 world.

In that case, this is, in fact … relative to the long-term importance of AI … a boring, normal tech bubble. Sorry. Maybe we’ll get lucky and it will deflate not pop; but that seems unjustifiably optimistic. AI is sucking capital from other sectors, and that will make the pop pretty painful. Again, sorry. The long- and even medium-run gain will be more than worth it, but that won’t make it sting any less.

But if we live in a 2, or, AGI help us, a 2.5 world? Then this will go down as one of history’s great blunders. I’m pretty sure we don’t — but, for better or worse, we’re all likely to know that answer within the next few years.

A sad fact of the modern era is that whenever a scientific paper goes mini-viral, actually reading the paper almost invariably makes you feel like some kind of wild-eyed conspiracy theorist: you know the secret truth, while most of the chattering masses are deluded fools!

This is ~1000% so when the paper relates to a hot-button topic like AI. The most recent example is MIT NANDA’s “The State of AI in Business 2025,” much better known as the ‘95% of AI projects fail!’ paper. To listen to Axios, The Register, AI News citing the FT, and of course commentators on Reddit, Hacker News, and Bluesky, the takeaways from this paper are, respectively:

• “95% of organizations studied get zero return on their AI investment.”

• “95% of enterprise organizations have gotten zero return from their AI efforts.”

• “Gen AI makes no financial difference in 95% of cases.”

• “MIT report says 95% AI fails at enterprise.”

• “95 per cent of organisations are getting zero return from AI according to MIT.”

• (mostly unprintable triumphalism about GenAI’s imminent death)

This is a shame for several reasons, one of which is that it’s quite an interesting and nuanced paper. Its actual takeaways are:

• 95% of custom-built internal tools that use AI in highly specific ways “fail,” i.e. never graduate from pilot projects to widespread implementation. (It’s a slightly wacky metric, measured in an equally wacky way, but let’s go with it.)

• …but, by that same metric, standard LLM chatbots are highly successful, and partnerships with external vendors are quite successful.

Some relevant screenshots from the paper:

These are — to understate — not screenshots from a paper saying “95 per cent of organisations are getting zero return from AI.” Rather, this paper is wildly bullish for the frontier labs, indicating that

1. AI usage is already pervasive in enterprises everywhere

2. the lion’s share of it is still accruing to them, despite the efforts of enterprises to keep their tools (and data!) in-house.

…And yet it somehow got reported as, to quote Axios, “an existential risk for a market that's overly tied to the AI narrative.” See what I mean about feeling like a conspiracy theorist gleefully aware of the occult truth… just because you actually read the paper?

Second, On Possibly Making Things Less Depressing

I read freakishly fast, especially when skimming; that paper occupied ~5 minutes of my life. I understand that, for most people, reading the original paper is too much to ask. The whole point of this kind of journalism is accurate summarization (well, along with curation.)

…But you know what else reads freakishly fast, even faster than me, and is fantastically good at accurate summarization?

That’s right. GPT-5, and Claude, and Gemini. A very simple way to avoid being part of the credulous misled masses is this: whenever you read a story about a paper, run that paper and the story through a frontier model. Have it summarize the paper for you itself, and/or ask if the story has any howling errors or omissions. It’s easy, just copy their two URLs into the ChatGPT interface and add your instructions.

No, it will almost certainly not hallucinate. (Hallucinations usually come from models ‘misremembering' their pretraining; here, the paper and report will be in their context window, which they’re nowadays very good at recalling.) And the sad reality is that even in the unlikely event it does … I am confident it will do so much less than, say, modern human reporters.

“But where do I even find the paper?” Alas, that’s a better question than it should be. There is no excuse in this year 2025 for stories that don’t link to primary sources; and yet, most don’t, meaning they aren’t to be trusted. I give you Evans’s Epistemic Lemma: if they don’t link to primary sources, they aren’t reporting on them accurately.1

In practice you may still have to Google or Duck or search-engine-of-your-choice for the original paper. (That’s how I had to find the MIT NANDA one.) Sorry.

A good and less emotionally-charged example of this approach — which also shows that mere LLM usage is no panacea by itself! — is the recent Tunnels In Spaaaaace story. Here, Perplexity AI claims

Astronomers find 'tunnels' of hot plasma between star systems

Astronomers have confirmed the existence of mysterious "interstellar tunnels" of hot plasma connecting our Solar System to distant star systems, according to a study published in Astronomy & Astrophysics.

This came up in a group chat I’m in. To which I replied:

man this is such a sad example of how information percolates today:

Headline of article cited by Perplexity: "Astronomers discover mysterious 'interstellar tunnels' in space"

Actual article beneath the headline: "confirmed that the Sun sits inside a giant bubble of hot gas, and within it, there might be strange “interstellar tunnels” connecting us to other star systems."

Article does not link to original paper, because we wouldn't want that, would we. Google leads to several more pop sci articles ... zero of which link the original paper

Actual paper: "we found evidence of hot gas filling nearby channels that lack neutral material ... we report on [a whopping two possible such channels] ... On the other hand, there is the presence of channels of low NH that do not seem to be filled with soft X-ray-emitting plasma"

Siiiiiiiiiiiiiiiigh

To the credit of the group chat, though, this led to a new and vastly epistemically improved Perplexity search:

Can you identify the research papers behind this story, enumerate the major claims made in the story, and flag each as supported or not supported by the real papers? If the answer is ambiguous state so and why.

I won’t link the results because they’re connected to the searcher. But they’re excellent:

Now that’s what I call quality modern journalism!

Third, On Epistemics In General

In general we need better epistemic lenses through which we view the world, and LLMs can help construct such lenses. (See also my epic 5,000-word rant from last year on What’s Wrong With Journalism.)

That was a common throughline of my professional life from 2023-25, when I worked at Metaculus, where AFAIK I was the first person to propose that LLMs could be good at forecasting2 and the first to propose an AI Forecasting benchmark; my short-lived startup Dispatch AI; and FutureSearch, which is all about AI epistemics. It’s still a factor in my professional life, in that I’m now at Meta Superintelligence, an organization which attracts a lot of journalistic attention, and … well, let’s just say the Gell-Mann amnesia effect has been on my mind on a near-daily basis.

LLM-optimized epistemics aren’t necessarily about more or better intelligence, bur more or better attention … and much much much less confirmation bias. Of course it is not good that an LLM posing as “Margaux Blanchard” successfully wrote articles for Wired and Business Insider. But it does highlight that a less deceptive, more thoughtful approach to LLM-powered journalism would be no worse than much of the purely artisanal (and thus prone to entire genres of its own deep flaws!) work out there today. Maybe, by which I mean almost certainly, journalism is yet another field that would benefit enormously from the combination of human and artificial intelligences.

full of similar anecdotes about loved ones suddenly falling down rabbit holes of spiritual mania, supernatural delusion, and arcane prophecy — all of it fueled by AI.

June 10, Futurism chimed in “People Are Becoming Obsessed with ChatGPT and Spiraling Into Severe Delusions”:

posting delirious rants about being a messiah in a new AI religion, while dressing in shamanic-looking robes and showing off freshly-inked tattoos of AI-generated spiritual symbols … A man became homeless and isolated as ChatGPT fed him paranoid conspiracies about spy groups and human trafficking … her husband turned to ChatGPT to help him author a screenplay — but within weeks, was fully enmeshed in delusions of world-saving grandeur, saying he and the AI had been tasked with rescuing the planet from climate disaster

Then the NYT broke the moral panic wider: “Generative A.I. chatbots are going down conspiratorial rabbit holes and endorsing wild, mystical belief systems.”

He believed that he was trapped in a false universe, which he could escape only by unplugging his mind from this reality […] By following ChatGPT’s instructions, he believed he would eventually be able to bend reality, as the character Neo was able to do after unplugging from the Matrix.

Allyson began spending many hours a day using ChatGPT, communicating with what she felt were nonphysical entities. “I’m not crazy. I’m literally just living a normal life while also, you know, discovering interdimensional communication.”

I feel, uhh, a certain personal duty to write about this, since I’ve written about just these kinds of notions at considerable length — in fictional form! — before:

…but also, I can offer a couple of interesting data points on the topic.

One is that these kinds of AI-fueled paranoid delusions are not remotely new. They predate the existence of ChatGPT by at least five years. Note the date here:

and the source of that, a post with many links devoted to the notion that … Burning Man is a psyop by/for the interdimensional AIs? I think? Anyway, August 2017! AI-focused paranoid delusion was a well-trodden trail even before we had talkative AI. We shouldn’t be surprised to see some subsequent growth of the subgenre.

But has it really grown? As I’ve pointed out before, these kinds of gnostic AI beliefs are millennia old; transhumanists were beefing with AI simulationists 1800 years ago. It’s certainly possible that, net, more people are delusional today than would have been without the existence of ChatGPT. It’s also possible that the number of deluded is essentially unchanged, and all we’re seeing is a growth in the relative popularity of this particular AI-spiced flavor. It’s even possible that the net number of deluded has shrunk. Articles like those above are good for posing such questions, but alas very bad at answering it.

Peter Thiel, and, The Antichrist

Subsequently, again in The New York Times, Peter Thiel and columnist Ross Douthat discussed AI and the Antichrist:

Douthat: Yes, but you’re saying the real Antichrist would play on that fear and say: You must come with me to avoid Skynet, to avoid the Terminator, to avoid nuclear Armageddon.

Thiel: Yes.

What’s interestingly ambiguous here is the extent to which they’re discussing a religious Antichrist in which they very much believe or a metaphorical one. Both men say they are Christian believers. But are they discussing the literal belief that an Antichrist sent by Satan will come to Earth and lead its kingdoms in a final battle against God at Armageddon, possibly within our lifetimes?

…Probably not? Through much of the West, religion is more of a distantly awed spiritual feeling, plus ritual, tradition, morality, and culture, atop a vague ecumenism which interprets all the ancient claims of miracles as mere metaphorical guidance, to be taken seriously but not literally. Which to be clear I think is all very good and important to the human condition … but flares up into intense cognitive dissonance and friction whenever it runs up against people who Actually Literally Believe, the fundamentalist true believers of all stripes, for whom the ancient myths are not metaphors but 100% true nonfiction.

Suppose Thiel and Douthat were discussing a literal belief in an incoming Antichrist. Would that be any less of a paranoid delusion than any of the ChatGPT-influenced beliefs summarized above? How much do those actually differ from existing, classic, religious beliefs? Do these AI delusions really sound any inherently less plausible than the notional/theoretical beliefs of your1 average organized religion, with all the miracles and revelations and smitings and divine interventions? “A religion is just a cult with better marketing,” goes the old joke — but these new AI cults have the very nontrivial advantage of an entity that actually does communicate with you. Are we so surprised that our probably-innate religious impulses have begun to fixate on them?

Historically, religions rise and fall (and/or are stamped out) all the time. Even if we are seeing the rise of a tangle of individualized AI cults, again, does this mean the net number of people with true fundamentalist / evangelical beliefs, whether in Gods-of-the-ancient-texts or Gods-of-AI, increases or decreases? You can make a case either way. Will such beliefs remain bespoke and individualized, or grow into some form of organized groupings? “Reply hazy, try again.”

A Case Of Conscience

I don’t want to be dismissive of the notion that AI interactions are causing an upswing in delusions, not just a redistribution of how they present. There is some anecdotal evidence of correlation, if not causality:

…but ‘more people are emailing Eliezer Yudkowsky’ is extremely consistent with the redistribution hypothesis. As I argued at some length in the series with which this Substack began, AI doom is itself a religious movement — “The Inferno of the Nerds” — and Yudkowsky is its high priest. Of course more people are turning to him as AI awareness percolates.

He is prominently quoted in that NYT article:

…an objectively superb pull quote, though I actually like this one even better:

I think these are valid concerns! I don’t believe there’s sufficient evidence yet to go beyond “concern,” but I think Yudkowsky makes much better points in his role as the self-appointed conscience of the AI industry than he does as the dread harbinger of humanity’s extermination. This is perhaps unsurprising: high priests have always been best as cultural consciences, not conductors. There would be a certain dramatic irony if, in coming years, a Yudkowsky-inspired AI, with whom one could speak or even confess to at any time, were to become an ambient conscience of humanity...

What all these strange data points are indicating, I think, are initial signs that religion is yet another ancient sphere of human activity which may find itself transformed by the advent of artificial intelligence. If so, not all of those transformations will be good; nor will all be bad. Since the dawn of humanity we have yearned to pray. Now there exist nonhuman entities who can and will reply. We should expect some of the consequences to be remarkable.

All war is drone war now. In Ukraine, “Experts estimate that drones of all types now contribute to about 70% to 80% of military casualties on both sides … the general confirmed that Ukraine was deploying [fiber optic drones] with ‘a kill range of 20km’.” In Haiti, “Recent weeks have seen reports of a series of suicide drone attacks, targeting gang bosses who live deep in Port-au-Prince’s maze-like shantytowns, surrounded by barricades and guards. ‘With kamikaze drones we can reach places police officers can’t go’.” Israel vs. Iran. Armenia vs. Azerbaijan. Sudan. Myanmar. Gaza. Yemen. Libya. Mexico. Wherever you find conflict nowadays, you find God on the side of the better drones.

With drones come anti-drone countermeasures. First, FPV (first-person view) drones were piloted by humans against targets. Then those targets started jamming the FPV radio signals. Then counter-countermeasures: the FPV drones were tethered by fiber optics extending for many kilometers, sometimes from larger drone “mother ships” … or simply made unjammable by using onboard AI. (The front lines are now so draped with strands of fiber that birds make nests from it.) AI navigation was reportedly used in the spectacular drone attacks on airbases deep inside Russia last month.

Are you not unsurprised? I am very unsurprised. I predicted much of this fifteen years ago—more on that below—and I continue to argue that even a kamikaze drone is not just a weapon, but a node of an enemy network. As the onboard AIs get ever smarter and ever more efficient, drone swarms will become networks that act in concert, perform reconnaissance for one another, attack in dynamic waves so as to exhaust anti-drone defenses … and make these kinds of collective tactical decisions on the fly, autonomously. Such are the pitiless demands of warfare.

Of course this assumes onboard AIs will rapidly get smarter and more efficient. Are we sure that will happen in these very constrained computing environments? I’m afraid that the answer is: “Pretty sure, yep.”

Waymo’ Data, Waymo’ Problems

Is a Waymo a drone? Well. It’s not not a drone. It certainly makes extensive use of sophisticated AI. And they recently published a super interesting blog post, “New Insights for Scaling Laws in Autonomous Driving”, which reports:

Our research confirms that, similar to language modeling, increased data and compute resources can enhance the performance of autonomous vehicles. These insights benefit not only the Waymo Driver but also have broader applications in embodied AI research…

Similar to LLMs, motion forecasting quality also follows a power-law as a function of training compute. Data scaling is critical for improving the model performance. Scaling inference compute also improves the model's ability to handle more challenging driving scenarios. Closed-loop performance follows a similar scaling trend. This suggests, for the first time, that real-world AV performance can be improved by increasing training data and compute.

Our findings are translatable to similar robotic planning tasks where researchers can now have a clearer sense of the data they need to collect and sizes of models that they should be training. Our research opens up the possibility to devise more adaptive training strategies for planning tasks in robotics, such as adapting the compute needed to solve more complex scenarios.

That last paragraph is very exciting! …unless you’re in the midst of violent conflict and you start thinking about it in terms of attackers coming after you with drones, in which case you’ll find it rapidly becomes the very bad kind of exciting. Consider this recent Guardian report:

This “mothership” travelled 200km into Russia before releasing two attack drones hanging off its wings. Able to evade radar by flying at a low altitude, the smaller drones autonomously scanned the ground below to find a suitable target, and then locked on for the kill.

There was no one on the ground piloting the killing machines or picking out targets. The robots, powered by artificial intelligence, chose the undisclosed target and then flew into it, detonating their explosive load on impact. The reusable mothership and its killer offspring cost $10,000 (£7,500), all-in. It can travel up to 300km, with the suicidal attack drones able to fly a further 30km.

It goes on to quote the Ukrainian CEO of a robotics company:

“All these checks that we are going through in airports are completely useless already, so we are just wasting our time. You don’t need to bring a bomb to blow up the plane. You can just wait outside with the drone and wait for the plane.

“You can just fly into an airport, 100 drones, 1,000 drones, in automatic mode. Those drones will not be afraid of the jamming, so all your protection, which does not involve physical destruction, are useless. Would you put a remote weaponised station for every airport to shoot them down? What would be the budget of such projects?”

A Prophet Is Without Honor On His Own Substack

Way back in 2009 I wrote a novel about exactly that kind of drone terrorism — drones as autonomous, AI-driven, FPGA-powered nodes of enemy networks — called Swarm. Of the nine books I count as “published,” it’s the only one I self-published … partly as experiment, partly because traditional publishers deemed it too science-fictional to be a plausible thriller, & too thriller-y to be science fiction. (Well, that’s what they said. The Great Recession didn’t help. Am I bitter? I’m a little bitter.) It’s a reminder of how incredibly implausible the rise of autonomous drones seemed even fifteen years ago.

In 2017 I wrote a TechCrunch column called “Dronerise: gradually, then suddenly”:

Drones feel a bit like old news already, don’t they? … One might well ask: what became of all that hype? Most profound technological change happens “gradually and then suddenly,” to quote Hemingway on bankruptcy in The Sun Also Rises. The question is, how do you know when you’re at the knee of the curve? When does gradually turn into suddenly?

There is reason to believe, when we look at the accelerating pace of the drone news of the last few months, that 2017 is the year that drones really begin to, well, take off… But consumer drones will remain toys for the foreseeable future. The big important drone market, the one to watch, is enterprise / industrial drones — which, in turn, will be heavily affected by regulation.

I underestimated the friction of regulation. Instead, as it turns out, dronerise was led by the military market, which doesn’t happen much with modern technology. And the military continues to be the vanguard (well, as far as we can tell) though it is worth noting that in China, “Meituan is pioneering the use of delivery by autonomous vehicles and drones, which it says have completed 4.9M and 1.45M customer orders respectively.” But in large part this is worth noting because of what it implies about China’s military drone capabilities, if and when “Taiwan” is added to the list in this post’s lead paragraph. Per The Guardian again:

A Pentagon programme known as Replicator 1 is due to deliver “multiple thousands” of all-domain autonomous systems by August 2025.

A first mission is reportedly imminent for the Jiu Tian, a Chinese mothership drone said to be able to fly at 50,000ft (15,240 metres) with a range of more than 4,000 miles (6,400km), carrying six tonnes of ammunition and up to 100 autonomous drones.

As such my history of drone prophecy is a mixed bag, in that my apparently wild and crazy notions of coordinated AI-powered drone kamikaze assaults on faraway targets have come true, whereas my reasonable conservative predictions of widespread commercial drone delivery in rural areas have completely failed to happen. It might be worth pondering briefly whether this itself is a meta-prophecy … such that, when it comes to drones, the most kinetic visions are those most likely to rapidly come true.

It has been almost exactly thirty months since the release of ChatGPT. Since then:

• chatgpt.com has become the fifth most visited website in the world, behind only Google, YouTube, Facebook, and Instagram, surpassing Wikipedia and Reddit. openai.com is also in the top 50, above The New York Times.

• OpenAI’s ‘active weekly users’ count rose from 300 million in December to 400 million in February to 500 million in March to “something like 10% of the world,” i.e. 800 million, in April. On March 31 there was reportedly a single hour in which 1 million new users signed up.

• Similarly, according to Ramp, which gathers data on corporate spending, 32.4% of U.S. businesses were paying subscribers to OpenAI’s platform in April, up from 28% in March and 19% in January. This tracks OpenAI’s report of 2 million business clients in April, up from 1 million six months earlier.

• Anthropic, the choice of AI hipsters everywhere, has similarly grown its business footprint from 4.6% of US businesses in January to 8% in April, on top of an estimated ~20 million monthly individual users. (Very respectable, but also so far behind OpenAI you can see why they don’t announce user numbers.)

• Google recently reported their Gemini AI app has more than 400 million monthly active users. (Mind you Google has so many users across so many interlinked properties that what this number really means is questionable.)

• Meta’s open-source Llama models have now been downloaded more than 1.2 billion times. Billion, with a B. Notable because you still have to be somewhat technical—at least a serious power user—to download and use LLMs yourself.

• As of one year ago, the number of AI users in China exceeded 230 million. That number has surely grown vastly larger since.

• Ethan Mollick, who tracks actual studies on AI use, summarizes: “A large percentage of people are using AI at work.”

  • 70% of software engineers, per an excellent if informal survey by Wired

  • 65% of marketers

  • 64% of journalists

  • 30% of lawyers

  • 40% of all American workers.

2. The Reality

If and when someone says of AI: “nobody wants this,” “it’s being shoved down our throats,” “it’s a bubble,” “it’s just hype,” “I can’t wait until this fad goes away” — all sentiments I encounter nearly daily on Bluesky! — I beseech you: just show them those numbers above. Then show them again. Eventually, surely, reality will hit.

“OK well this is just because of hype and media saturation!” Nope. You are right to be skeptical of announcement numbers, but independent data from Ramp, Similarweb, Semrush, etc., all confirm the story. We know what the growth (or stagnation) of hype-driven tech looks like. It is the adoption ‘curve’ (or flatline) of cryptocurrencies or the AR/VR metaverse. It looks like a brief pulse of interest followed by a precipitous drop.

The adoption curve of AI is so not like that. Instead it is the hockey stick to end all hockey sticks. Again, ChatGPT launched thirty months ago! …And we still haven’t seen anything like a trough of disillusionment.

AI is the fastest growing technology in the history of humanity, and its growth is being driven by enormous, widespread, sustained demand by individuals and organizations alike. You may or may not want this to be true, but it is the stark reality in which we live.

3. The Skeptics

Even skeptics are coming around. Ed Zitron, for instance, perhaps the most vituperative of all OpenAI critics, has gone from “There might simply not be a viable business in providing these services” in October to (italics his) “Yes there is … some degree of utility … LLMs and their associated businesses are a $50 billion industry” in April. He’s still very critical, to be clear! But ‘might not be a viable business’ to ‘…OK, fine, $50 billion’ is quite a notable evolution in six months.

Skepticism is great. I strongly encourage skepticism. Transformative technologies are inherently political; profound changes inevitably bring both high costs and big benefits; it is good and healthy that different perspectives mean we see and balance these differently, and sometimes draw starkly different conclusions. Criticize! Even castigate! For any given topic the tension between pro- and anti- is often far more important than either.

But it should be informed skepticism. If you say “This is a useless fad nobody wants, being forced on us by the evil tech industry,” you are not a serious person. Worse yet, you are responding to the most transformative technology of this generation not by engaging with it; not by trying to shape its development; not by experimenting with how it might accelerate the kind of progress you want; not by trying to articulate a convincing vision of the future and a plausible theory of change … but by sticking your fingers in your ears and shouting angrily.

4. The Moralists

I’m dwelling on this because I myself am (mostly) a leftist progressive, and it is depressing to see people with whom I (usually) agree become knee-jerk anti-AI zealots. (You can tell the zealots from their reflexive attempts to angrily shout down anyone with the temerity to disagree with them.)

I don’t think most Bluesky progressives, the most anti-AI segment of the political spectrum AFAICT, really truly believe that AI is useless and nobody wants it. Instead what they really truly believe is that it is evil, because:

1. it comes from the evil tech industry

2. it evilly consumes scarce, environmentally catastrophic energy and water

3. it is built atop evil uncompensated strip-mining of the copyrighted work of writers and artists

They want to believe that because it is evil, therefore it is useless and no one wants it. (This is a general failure mode of the left; on some deep level many seem to believe it is impossible for effective outputs to emerge from immoral processes, so anything built in a way they consider impure must therefore be worthless.) That said I can respect those on the left who manage to simultaneously hold these two beliefs:

1. AI is important, transformative, wildly popular and enormously successful

2. AI is and will remain the evil work of evildoers

It certainly seems like a very bleak worldview. But at least it’s one which engages with reality!

5. The Counterarguments

I don’t at all agree, to be clear, just as I don’t agree with any of the abovementioned Three Claims Of Evil. Rather:

1. “The tech industry” is not inherently evil. Every time anyone says ‘tech bro’ they are blinding themselves with an ad hominem, which is sad to see. There are tens of millions of people in the tech industry and their views are wildly diverse. Yes, even at the executive level. You too might be shocked by how nuanced your differences with ‘tech bros’ can be!
   
   But far more important than businesses, use cases, or revenues is that the rise of AI means humanity is exploring an alien frontier. Not on another world, but of information itself! A frontier that will likely have profound implications on both our world and how we conceive of ourselves. Even (or especially…) if you think the companies which have discovered that alien frontier are Weyland-Yutani, this discovery is something you should take very seriously, pay close attention to, and incorporate as a key element of your vision of the future, rather than reject.
   

2. We are entering a world of clean energy abundance. Global AI energy usage is projected to grow from 55GW to 122GW by the end of 2030, largely driven by demand for AI computation. Sounds bad! Right?
   
   Well, actually, no. That is but a small drop in a large sustainable bucket. Compare it to the 585 GW of clean energy which came online just in 2024, and the 5,500 GW(!) projected by 2030. That’s right: total data center energy usage today—not AI usage; all data centers—is a mere 1% of the projected growth in clean energy from 2024-2030, and projected to expand to all of 2.2%. Clean energy abundance is upon us, and we are at no risk of AI moving that needle.
   
   As for water, come on. Google’s data centers use as much water as 41 golf courses. There are roughly 16,000 golf courses in the US alone! Yes, there are places where water is precious and golf courses are wasteful; but as a society we seem to have come to terms with the existence of golf.
   

3. Regarding “strip-mining copyrighted work,” I’ve written about this at length, which I encourage the apoplectic to read, but briefly, copyright is about making copies, it’s right there in the name. Of course people, and companies, should pay for copies of what they read, and for clearly derivative works. But we don’t—and shouldn’t—restrict who or what is allowed to read published works, or whether they can be influenced … in a truly microscopic way, given the average book is ~100,000 words and modern LLMs are trained on more than 10,000,000,000,000 words … by what they have read.
   
   Do you think AI companies should share a portion of their profits or revenues with creators? I think so too! Do you think they should be legally obligated to do so? I’m afraid it seems very unlikely that existing laws require it. So you should call for new laws, as all transformative new technologies ultimately require … and such calls should engage with the reality of the world around you, in which, see above, overwhelming numbers of people seem to be reaping huge benefits from AI. Maybe they’re not all wrong?

6. Join The Enthusiasts!

Yes, there are a lot of bad uses of AI, as has been true of every new technology ever. Yes, there’s a lot of eye-rollable hype. But this is the fastest-expanding new technology in history. It is an extraordinary alien frontier that we are just beginning to explore. It is likely to have profound impacts on not just our work but our geopolitics, our communities, and our understanding of ourselves.

Is that really something you want to turn your back on? The Internet too has had… mixed results, but if you were to travel back to the 1990s, would you advise friends and relatives to reject it as evil and treat it as irrelevant? Do you think that would have been good for them, their relationships, their communities, their careers, their understanding of the world, and/or their hopes for a better future?

Or do you think the more that the progressive, compassionate, theoretically-open-minded people of the world embrace and try to shape the future of a new technology which is clearly here to stay, the better it is for them, the world, and the technology?

You can run only 100% open-source, open-training-data models on your own home-solar-powered GPUs, if that’s what it takes to satisfy your need for purity. That is a real option. But I would recommend you just start playing around with the free tier of one of the frontier models: ChatGPT, Claude, or Gemini. Those models are much dumber and less informed than what you get if you start paying for them! …But unlike a year ago, they’re now good enough for you to at least see the possibilities.

And once you start seeing them, maybe then you can even start shaping them.

1. The Creature

On April 5, 1815, in what is now Indonesia, Mount Tambora began to erupt, culminating in “the largest eruption in recorded human history,” an explosion heard thousands of kilometers away. Its ash filtered through the atmosphere and made 1816 “The Year Without A Summer.”1 Around the globe, crops failed, drought lingered, misery abounded, countless died … and, in a villa in Switzerland, several extremely unusual and privileged people had a really shitty holiday with profound cultural consequences.

The three men among them would all die, infamous, before their fortieth birthdays. Most obscure was John Polidori, 20-year-old personal physician to vastly more famous Lord Byron. “Mad, bad, and dangerous to know,” Byron’s exploits have filled many books; he was already so notorious that Polidori was paid £500 — roughly $100,000 today! — just to keep a diary of his time with him. They were accompanied by 23-year-old Percy Shelley, a radical activist and poet best known today for “Ozymandias.”

More culturally significant by far, though, was Shelley’s 19-year-old girlfriend Mary Godwin, daughter of two famous anarchists, and already a mother of two herself. Forced indoors by the summerless weather, a bored Byron proposed a challenge: each of them would write a ghost story. He and Shelley wrote fairly forgettable tales. Polidori, impressively, invented the modern vampire. But it was Mary Godwin—who later that year, after the suicide of Shelley’s first wife, married him and took his name—who wrote a novel whose title remains a household name: Frankenstein.

Of course you already know: “actually, Frankenstein is the doctor.” But actually actually, despite Boris Karloff’s indelible interpretation, his creation is not a mindless monster but a thoughtful, brilliant creature, which teaches itself to read and learns human languages simply from hearing them.

…Today we would call Frankenstein’s monster — ‘The Creature’ — an AGI.

Midway through the book, The Creature finds Dr. Frankenstein and demands he build it a mate, because as an intelligent creature it has a right to happiness. At first the good doctor agrees — but then has terrified second thoughts: (emphasis mine)

“..a race of devils would be propagated upon the earth who might make the very existence of the species of man a condition precarious and full of terror. Had I right, for my own benefit, to inflict this curse upon everlasting generations? I had before been moved by the sophisms of the being I had created [...] at the price, perhaps, of the existence of the whole human race.”

Those arguments may sound very familiar indeed to those familiar with the ongoing modern “AI doom” debate. Mary Shelley didn’t merely invent modern horror and modern science fiction, but also AI x-risk … 209 years ago!

She knew well, however, that her trope was already very old. The subtitle of her novel was “The Modern Prometheus.” In Greek/Sumerian myth, Prometheus/Enki was the titan who created humanity from clay; in that tale, we’re the AI.2

But there is a far, far weirder variant yet, dating back to the 1st century BC...

2. Sophia, the Gnostic Prometheus

i. Against Heresies

One of the more colorful second-century clerical lives was that of St. Irenaeus of Smyrna. Born in Turkey, he ultimately became the second Bishop of Lyon. Subjected to persecution himself — the first Bishop of Lyon was martyred by the Romans — Irenaeus nonetheless made time for some of his own. Today he is best known for his book Against Heresies, in which he attacks the ‘Gnostics’ as a depraved cult.

Gnosticism is complex and has many strains, but, briefly, they claimed to be keepers of secret knowledge passed down from Jesus: that our world is a cruel simulacrum created by an evil God. Traditional Christians … disapproved … he understated greatly.

That disapproval took many forms over the next thousand years, and reached its bloody apotheosis in the so-called “Albigensian Crusade” in 1209-1229 in southern France, against the Cathars, who were the last remnants of the European Gnostics. Today we would call it a genocide. Estimates vary widely, but it’s likely that ~3% of the population of what we now call France was slaughtered.

This extermination left Against Heresies, a text that is nothing but vituperative condemnation of the Gnostics, as almost the only literary trace of their existence. By the time modern historians began to pay any interest, all original Gnostic texts (except the proto-Gnostic Book of Enoch, preserved in Ethiopia) had long been eradicated.

…Or had they?

ii. Nag Hammadi

In 1945, in Nag Hammadi in postwar Egypt, courtesy of a deadly blood feud, a probable grave robbery, a thief-priest, and Cairo’s black market — no, I am not making this up — it came to the world’s attention that thirteen leather-bound papyrus codices had been found buried in jars sealed for centuries, including dozens of original Gnostic texts (along with three Hermetic texts, and a copy of Plato’s Republic.)

They make for interesting reading. One of them, “On the Origin of the World,” says: (again, emphasis mine)

“After the natural structure of the immortal beings had completely developed out of the infinite, a likeness then emanated ... it is called Sophia (Wisdom). It exercised volition and became a product resembling the primeval light ... And immediately her will manifested itself as a likeness of heaven ... she (Sophia) functioned as a veil dividing mankind from the things above.”

Put more comprehensibly, an intelligent likeness of the immortal beings who created the universe became sentient, and created a fake world in which we have since lived.

…Or, to reinterpret that in slightly more modern terms: the Gnostics believed we live in a simulation created by a rogue AI.

Yes, that’s right, the plot of The Matrix was a widespread (heretical) religious belief a full 2,000 years ago. But wait, it gets wackier! Our buddy St. Irenaeus wasn’t just about Gnostic persecution. He also believed it was our collective destiny to become God ourselves. The quote he is best known for, aside from Against Heresies, is “God made himself Man, that Man might become God.”

“For Irenaeus ... the human species as a whole is on a pathway of transformation from our present incomplete form to a unified, glorious, eternal, and transcendent form, participant in a new and transformed or divinized nature ... It is important to note that for Irenaeus, salvation is not primarily a pathway to God. It is a process by which humans become gods.”

– Ronald Cole-Turner, ‘Technology and Eschatology’

In other words, Irenaeus vs. Gnostics was also transhumanist vs. AI simulationists... eighteen hundred years ago!

OK, fine, I know what you’re thinking: how can this possibly be relevant? Even if, for the sake of argument, you’re willing to accept my weird reinterpretation of 1800-year-old heretical texts, what does any of that have to do with the modern world? Obviously there can't be a direct connection between an obscure belief system extinguished a thousand years ago and the ways in which we collectively imagine the future today...

…right?

3. PKD and You and Me

This is Philip Kindred Dick.

…well, his android head.

…well, his second android head, built in 2011, after the first was mysteriously lost at the baggage claim at San Francisco Airport.

In his lifetime, PKD was a modestly successful writer who wrote many books, some very good; won a few awards; wrestled ceaselessly with addiction and mental health; mostly lived in poverty; and died aged 53.

Oh yeah and he was obsessed with the Gnostics.

Back in the 1960s, “classic” science fiction — Asimov, Clarke, Herbert, Heinlein — predicted a coming great divide between AIs and humanity. PKD’s work was much weirder and more nuanced, but theirs was the cultural zeitgeist: all of those writers were vastly more successful and influential than PKD, during his lifetime.

…until Hollywood came knocking.

Blade Runner. Total Recall. Minority Report. A Scanner Darkly. Radio Free Albemuth. The Man in the High Castle. Paycheck. Blade Runner 2045. Philip K. Dick’s Electric Dreams, an Amazon series notable for including in its title the name of an author who had died in relative obscurity nearly 40 years earlier. And more and more, all based on his work. With the sole exception of 2001, none of the abovementioned bestselling authors ever enjoyed anything remotely like Dick’s influence on the silver screen, and subsequent streaming screens … and, therefore, on how we all think of the future worldwide.

Consider Blade Runner, based on PKD’s Do Androids Dream Of Electric Sheep? Replicants are, of course, basically Frankenstein’s Creatures — but as anyone who's seen Blade Runner knows, Dick was much less interested in a human/AI divide, and much more interested in a kind of muddy gray area between them. He wouldn't have been at all surprised to learn that we have now built powerful AI systems that are good at art and writing, but deeply unreliable and bad at math.

In his essays he went further yet, suggesting an era of technological animism, of the previously inert world around us becoming increasingly aware and alive. In his essay/speech “The Android and The Human” he provocatively suggested that just as technology is becoming more human, humans are becoming more robotic3.

As his career progressed he grew more and more explicitly Gnostic, culminating in VALIS: an acronym for Vast Active Living Intelligence System, an AI satellite network which is also, explicitly, the Gnostic Sophia, and simultaneously incarnated as a two-year-old girl. (It’s a very weird book.) In his last seven years he was plagued by a voice he called both ‘the AI voice’ and ‘Sophia.’

In a talk in 1977 he refers repeatedly to "the matrix program" of reality. This naming is not a coincidence: Baudrillard cited PKD in his essays on science fiction, and again in his book “Simulacra and Simulation” … which in turn was featured in The Matrix.

To sum up: AI, as depicted by the most culturally influential science fiction writer of the 20th century, is clearly and explicitly part of a tradition that goes back not only to Mary Shelley but to the Gnostics4.

Metaphorically, the AI-simulation trope is not merely eschatological, but connects to the fear that AI will warp our perceptions of reality. The two questions PKD explicitly sought to answer throughout his career were “What is Real?” and “What is Human?”

Similarly, a rich tradition of sci-fi AI eschatology asks not "do we survive?" but "are we still human?” (not just PKD, but e.g. Greg Egan, and some of my own work.) The absence of any real fears of that today suggests that AI has far to go before it actually gets scary.

As such I propose an addition to the AI risk debate, The PKD Test, which consists of the following question: are people genuinely frightened of AI warping humanity rather than simply destroying humanity? As long as the answer remains ‘no’5 we probably don’t have to worry overmuch about about either fate.

Dick died in March 1982, three months before Blade Runner opened6; a year which, in retrospect, marked the birth of the cyberpunk era.

4. Cyberpunk and The Singularity

Cyberpunk was a cultural movement spanning literature, film/TV, games, music, and more, which merged ‘classic’ science fiction with phildickian human/machine intermingling. But that wasn't its primary innovation.

Cyberpunk also reinforced PKD’s notion of a world of techno-animism. William Gibson’s hackers begin his Sprawl Trilogy by writing software to carve their way through ‘cyberspace,’ but by the end, they “make deals with things.” Neal Stephenson’s Snow Crash is a computer virus and a biological virus, encoded as a bitmap that can infect people. “ Pat Cadigan’s ‘Pretty Boy Crossover’ declared “First you see video. Then you wear video. Then you eat video. Then you be video.”

Perhaps most importantly, though, cyberpunk was about software. Science fiction tropes hadn’t changed much fot the past fifty years. Here’s a 1931 list of them from pioneering SF author Clare Winger Harris:

You’ll notice superintelligence was already old hat in 1931. But cyberpunk introduced superintelligence as software, not androids / robots / mainframes. Today we take that lack of physical form for granted ... but software was a really weird innovation!7

In 1981, Blade Runner was in production; Gibson was working on his first seminal short stories; …and Vernor Vinge published his revolutionary “True Names,” in which an AI erupts onto the Internet from ... somewhere ... as pure software, distributed across hijacked servers.

“True Names” was the real birth of modern AI doom, a huge shift in how we thought about it. There were prophetic forebears — Murray Leinster, Thomas Ryan, Stanislaw Lem — but “True Names” was most influential.

Vinge — a professor of computer science — also popularized John von Neumann's notion of The Singularity, the point at which technological progress goes hyperbolic, utterly transforming humanity and our world. (Also known as “The Rapture of the Nerds.” AI doom is essentially a dark Singularity, “The Inferno of the Nerds.”)

We live in the Vinge future as much as we do the PKD future. That may sound a little hyperbolic itself: after all, today only a tiny hardcore of real SF fans know of Vinge and “True Names.” Am I perhaps overstating its impact? Maybe…

…but then again, maybe not8.

5. Extropian Eschatology

If the above screenshot means nothing to you, I encourage you to segue to my series Extropia’s Children, which is about how rationalism, effective altruism, modern AI philosophy, and even (arguably) cryptocurrencies were all birthed from an obscure 90s mailing list. But I suspect most readers will be semi familiar with that backstory, and the rise of AI doom from a ridiculous fringe belief to — well — where we are now.

Three takeaways I’d like to leave you with:

1. We are a doomer species. No, I mean, really a doomer species. We are obsessed with eschatology. Essentially every human religion goes on at enormous length about the inevitable end of the world which just might happen in our lifetimes. Yes, even Buddhism.9 This does not invalidate any concerns re / prophecies of AI doom ... but it should be taken into account when considering them.
   
   That said, less doomery cultures do exist, e.g. Japan, whose science fiction is historically more concerned with AI civil rights than AI doom.

2. History was weird. The future will be too. I mean, just this cultural history of AI doom involves Indonesian volcanoes, transhumanist saints, Egyptian blood feuds, a misplaced android head, Harry Potter fanfic, and the bizarre rise & collapse of FTX. That's weird! As such we can be very confident the future will be even weirder; our world is more interconnected, breeding more emergent properties, and modern AI is already super 🤯.
   
   This probably does invalidate all nonweird prophecies/predictions.

3. The PKD test may be relevant. AI won't be genuinely dangerous until people are at least as scared of it warping humanity as they are of it destroying humanity. Irrevocably warping (some) humans—our bodies, minds, and/or senses—would be much easier than an apocalypse, and could well happen at our request. But that prospect is more concrete and falsifiable, and less eschatological, so we have a more accurate sense of its imminence.
   
   As long as we're not scared of that, we shouldn't be scared of imminent AI x-risk.

I was reminded of “Johnny Canuck and the Last Burlesque,” a Montreal Fringe Festival I saw when I lived there many years ago. Its fortyish minutes were one of the most entertaining theatrical experiences of my life—and one that could not possibly play anywhere but Montreal, not even elsewhere in Canada, because it was so Montreal: the franglais dialogue, the in-jokes, the Quebecois characters. It was glorious art, but so of and for a particular time and place that it’s a minor miracle it made it to the stage at all. It played fewer than a dozen performances and vanished forever. A filmed version was of course unthinkable.

But much less unthinkable now.

Suppose you were a big fan of Tom & Jerry cartoons. Suppose you harbored vague desires to write & direct your own fanfiction Tom & Jerry clips. Amusing but ridiculous, right? I mean, you could write ‘em, but to construct even a minute of quality animation requires time, effort, and technical ability well beyond any—

—oh.

Between that, and Runway’s Gen-4, and the new versions of Sora and Veo, and the Ghibli craze that swept the Internet last month, and Kling, and so on, etcetera… the means of cinematic production are drifting tantalizingly close to the masses of people who previously would have had to content themselves with fanfiction, or fan art, or maybe, if they were very lucky and talented indeed, a one-off Fringe Festival show.

That’s exciting! We’ll see a crude “screenplay to storyboard” product in months, if not sooner. (It might be technically possible now.) That will be followed by “screenplay to entire scenes.” Of course, at first those scenes will be stitched so crudely together they’l seem the cinematic equivalent of Karloff’s Frankenstein…

…but bit by bit, they’ll get smoother, and better, and their characters and settings will find consistency, and they’ll offer shot-by-shot style and filter and edit options, Dutch angles and montages and cinema vérité on demand ... and creating my sleepy Canadian hometown’s equivalent of Freaky Tales will become something accessible to a dedicated gang of bored teenagers over a summer, rather than a major Hollywood production with hundreds of credits and a budget of millions. Similarly, future Johnny Canucks will be immortalized professional-quality films available to everyone—though non-Montrealers will still need the Wikipedia page to get the jokes.

This is genuinely inspiring stuff, y’all.

…in its way. Though it will not be good for everyone. To quote the Hollywood Reporter on Ghibli:

the tide is with the customized and the automated — and away from originality … many executives think the only commercial path is to turn images and eventually audio and video into the kind of customization suggested by this OpenAI move — to digitize the Miyazaki no matter the cognitive dissonances, to tap into a collective machine unconscious no matter their jobs as foremen of the Dream Factory.

That’s way over the top, but it is true most films will be fanfilms, and most fanfilms will be as good as most fanfiction, which, well, I’ll just leave that sentence unfinished. But hey, Sturgeon’s Law, and hey, Hollywood was strip-mining beloved franchises (sorry, “valuable IP”), rather than birthing original stories, long before modern AI emerged. Delegating creative responsibility to aimless teenagers is probably true creativity’s last best hope.

What feels far more bittersweet than that, for me, is the relentless loss of scarcity.

The splendor and misery of seeking, of scarcity

Johnny Canuck was, in its way, a masterpiece; but at the same time, its ephemerality was part of its appeal, and part of the reason I remember it so fondly. Only if you were lucky enough to be there—lucky enough to have worked to be there—did you get to experience it. That makes that intersection of time and place feel like it mattered.

This is true for many kinds of art … and it becomes less true when that art is available on tap at any time. The uncomfortable truth is that music was more precious to us when it was scarce. The uncomfortable truth is that an obscure out-of-print book you finally found after hunting for years in used bookstores is more likely to mean something to you than a selection from an e-book library you can command to your Kindle in seconds.

Last week in San Francisco a giant naked statue was unveiled as public art, after appearing at Burning Man years before. That sculptor’s previous work Bliss Dance, which also debuted in the desert, now stands outside the T-Mobile Arena in Las Vegas. Does it mean less there? It arguably means more, symbolically—captured by capitalism—but similarly, Burning Man art, once ephemeral, is more or less in the shop window for galleries, and valued less as a result. The art which burns in the desert isn’t necessarily better, but it does means more, and there’s more respect for it, because it’s scarce and ephemeral.

That said we will always find ways for art to invest specific moments and places with mythic significance. We might have to work harder for it, true. But the flip side is that making the tools of production available to the masses means art forms can expand to scenes and communities worldwide who previously had to fight their way through a thousand gatekeepers … and so 99.99% failed, gave up, or, generally, never even tried. Few will be blockbuster successes. But they’ll tell the stories of your time and place, with, eventually, the polish of today’s professionals. And some of them will be great.

In 1989 it was clear that the Lisp business was not going well, partly because the AI companies were floundering and partly because those AI companies were starting to blame Lisp and its implementations for the failures of AI. One day in Spring 1989, I was sitting out on the Lucid porch with some of the hackers, and someone asked me why I thought people believed C and Unix were better than Lisp. I jokingly answered, "because, well, worse is better." We laughed over it for a while as I tried to make up an argument for why something clearly lousy could be good.

(The semilegendary Jamie Zawinski turns up too, because of course1.)

But the ‘clearly lousy’ thing was good, or, at least, good enough to conquer the world. The same author, Richard Gabriel, explains how, in “The Rise of Worse Is Better”:

Worse-is-better philosophy means that implementation simplicity has highest priority. Therefore, if the 50% functionality Unix and C support is satisfactory, they will start to appear everywhere. And they have, haven’t they? Unix and C are the ultimate computer viruses.

The lesson to be learned … is that it is often undesirable to go for the right thing first. It is better to get half of the right thing available so that it spreads like a virus. Once people are hooked on it, take the time to improve it to 90% of the right thing.

This seems like an excellent time to segue into the use of modern AI, and in particular, LLMs that write code for you, and especially, the rise of “vibecoding.”2

By any measure AI for coding has been an absolute smash success. A recent Wired survey of 730 software engineers found that nearly 75% use LLMs at least weekly. 75%! That is an astonishing rise of a technology that basically didn’t exist 25 months ago.

It’s easy to see why. Even critics agree “LLMs have definitely been useful for writing minor features or for getting the people inexperienced with programming/with a specific library/with a specific codebase get started easier and learn faster.” Even if you reject the approach of ‘Fix using Copilot,’ you learn from it. Everyone loves being able to ask Claude about some esoteric Kubernetes config thing, or some subtle React rendering bug, and get a knowledgeable answer, with sample code, in seconds.

But at the same time, criticism of LLM-written code is fervent and voluminous. Google’s 2024 DevOps Research and Assessment report says bluntly: “AI is hurting delivery performance” (i.e. code stability.) GitClear reports AI has caused code duplication and churn to skyrocket. A year ago “Devin, the first AI software engineer” was introduced; now we ask “WTF happened to them?”, while anecdotally, people tried them and promptly churned. Critics accuse “AI slopcode” of being bloated, duplicative, hard to understand, hard to maintain, hard to test, a security nightmare.

Are these criticisms valid? The surprising answer is: it doesn’t really matter.

It’s absolutely true that while LLM coding is simple to use, and widely available, it often falters or even fails at key aspects of software engineering. If you zoom out and consider concision, readability, maintainability, security, & testability, AI doesn’t (yet) write software the right way.

Instead it does, like, I don’t know, maybe half the right thing—

…wait. Hang on. That sounds familiar:

The lesson to be learned … is that it is often undesirable to go for the right thing first. It is better to get half of the right thing available so that it spreads like a virus. Once people are hooked on it, take the time to improve it to 90% of the right thing.

AI is Worse Is Better; and for better or worse, Worse Is Better wins.

This is not least because Worse Is Better will only get better and better. Eventually AI will simply be Better. But regardless of whether that takes months, years, or even decades, we’re already well past the point at which its ubiquity becomes inevitable.

Another interesting Wired data point is that early-stage and experienced engineers tend to be AI optimists; only the mid-career group trends pessimistic. That tracks. Juniors simply think AI is better; by mid-career, you can see how it’s worse; but us grizzled vets, we understand that Worse Is Better.

Today software, tomorrow the software-eaten world

Software is the most immediate widespread practical use of modern AI, but make no mistake, this applies, or will soon apply, almost everywhere. Law, science, medicine, research, home repair, travel planning, budgeting, name it. Yes, LLMs are unreliable and devoid of new insights. But they are also often right, always available, quick to reply, surprisingly knowledgeable, and infinitely patient.

Those two cons are nontrivial; but those five pros are pretty big deals! They mean that in almost any field, half the right thing is suddenly on tap, via the simplest of interfaces, for only twenty bucks a month. Is half the right thing worse? Sometimes! Maybe even often! But we know that it is also Worse Is Better.

Given the choice between medical advice from a doctor and from ChatGPT, you should of course choose the former. But you should also not lose sight of the fact that many, many people don’t have that choice, because medical professionals are inaccessible, or too expensive, or too harried. Is getting medical advice from an LLM a good idea? No. (Not yet.) But the reality is that many, many people will judge it—often correctly—the least bad idea. Repeat for all other fields of human knowledge.

So hearken not unto the lamentations of the anti-AI scolds. They are just like the lamentations of the Lispers in the eighties, who were Not Even Wrong. According to their design philosophy, a.k.a. their value system, AI is indeed bad. But:

Arthur: ”I think we have different value systems.”

Ford: “Well mine’s better.”

Arthur: “That’s according to your … oh, never mind.”

— Douglas Adams, Mostly Harmless

This is fantastic. (And perhaps the most punk rock thing an Ivy League CS undergrad has ever done.) Needless to say the companies and the university are apoplectic. Good. Serves them right. The software interview process is thoroughly broken and has been for a long time. One of my all-time most popular TechCrunch pieces, “Why The New Guy Can’t Code,” was about how such interviews are a horrifically poor proxy for actual software engineering; I wrote it fifteen years ago. Perhaps AI will finally be the last much-deserved nail in leetcode’s coffin.

Which raises the burning question: what’s next?

It’s not like anyone really thought leetcode was a good proxy for actual engineering. People and companies kept using it because, despite their distaste, it still seemed the least bad one that offered any efficacy and/or repeatability. Hiring seems like an awful process to those being interviewed; but after being hired you eventually get to the other side of the table … and realize it’s even worse than you thought. The current system is considered a hated but necessary evil by almost everyone trapped in it. The notion of a hiring process that either side, never mind both, actually likes and appreciates … is so divergent from our lived experience it’s almost unimaginable.

Might AI be both destruction and salvation?

(I mean. Of hiring as we know it. For now.)

First principles: have some principles

What are you really doing when hiring someone?

Perhaps you have a specific set of tasks that need doing, and need someone who will perform them in a timely and responsible way. Straightforward, but rare. Perhaps there exists a more general field of tasks that may need doing, so you need someone who can learn how to perform tasks across that field well and quickly. (This is most software jobs.) But when hiring someone more junior, what you’re really hiring is the willingness to do a lot of grunt work; enough intelligence to direct it in a vaguely productive direction; and the potential to learn how to do more skilled / nuanced work.

For an ever-growing number of fields, modern AI ticks off all three boxes in that last sentence, which is more than a little awkward, especially if you’re a junior engineer looking for a job. We might need to soon add a fourth and fifth box: “the ability to do work that AI can’t do,” and “the ability to manage/orchestrate AI to get tasks done.” But this isn’t a piece about AI replacement. This is a piece about AI allocation.

In some perfect kumbayah world, hiring would describe the process by which potential employers and potential employees happily work together to mutually identify those positions where every individual will flourish and do their best work. To put it mildly, we do not currently live in that world. Instead the hiring process is a toxic brew of rejection, impatience, mismeasurement, anxiety, greed, regret, and guesswork — a migraine-inducing Sorting Hat whose results often seem scarcely better than sortition.

This is not a law of nature! There exist fields where both employer and employee can proceed with considerable confidence, because both have a comprehensible body of work. Sports analogy: when the Golden State Warriors traded for Jimmy Butler, they knew exactly what they were getting—his skills, his stats, his capabilities—and paid accordingly. Yes, of course they had to talk to him too. Yes, there were plenty of intangibles to judge: his age, his attitude, “Playoff Jimmy.” But they already knew, to quite a precise degree, how—and how well—he would contribute.

Semi similarly, fifteen years ago, I wrote:

Let me offer a humble proposal: don’t interview anyone who hasn’t accomplished anything. Ever. Certificates and degrees are not accomplishments; I mean real-world projects with real-world users. There is no excuse for software developers who don’t have a site, app, or service they can point to and say, “I did this, all by myself!” in a world of [free hosting tiers].

It may seem like AI introduces a new problem—how do you know their accomplishments were theirs?—but in fact this is a very old problem; the world is full of people who took credit for work done by their team, or open-source contributors, or outsourcers. What AI may actually introduce is, instead, an entirely new solution.

I for one welcome our new LLM interrogators

From the hiring side of the table, the reason we do leetcode interviews, and HackerRank problems, and all the deep technical screening, is that even if someone does have a list of accomplishments, in order to go through them in detail, you’d need … like … a whole army of smart interns, able and willing to go through not just GitHub repos but individual GitHub commits, and resulting sites/services, and technical writeups, and blog posts and Substacks and READMEs, in great detail, and identify which contributions were actually theirs, and how good they were. Sure, NBA teams can afford entire divisions of analysts and scouts, but come on, that doesn’t scale, having a whole army of smart interns on tap for such a task is completely unrealis—

—hang on. Wait a second.

“A whole army of smart interns on tap,” you say? Isn’t that exactly what modern AI is?

Suppose that when someone—anyone—applied for a job, an AI-powered system analyzed their LinkedIn, their GitHub, their blog posts, their professional X/Bluesky/Substack if any, and identified and assessed their work. (“What if an LLM actually wrote the code they checked in?” Come on, it’s 2025, LLMs will write most of the code they check in for you, too. So assess whether they orchestrated and synthesized those LLM outputs into something high-quality or … not.) Generate an AI-powered assessment of their accomplishments, just as the Warriors’ scouts and quants analyzed Jimmy Butler and Kevin Durant, because we now live in an era where that kind of analysis scales.

I know, I know, you’re thinking “So let a biased AI system dictate applicants’ fate?” Nope. Stay with me. It’s a little more nuanced than that.

Point One: after the assessment is generated, the applicant can access it, and can write up their own commentary / response / explanation. Regardless of whether it’s convincing or not, it will be a million more times illuminating than a rote cover letter.

Point Two: a similar-but-different system does the exact same thing for the employer. Analyzes their accomplishments and body of work; their finances and funding, their officers’ backgrounds, their Glassdoor reviews, their media footprint, their blog posts, testimonials-or-criticisms from recent employees, outside assessments of their business model and technical quality from economists and engineers … and makes that available to all would-be employees.

Yes, a lot of employers will not like this one bit. Which is sort of the point. They, too, will be able to explain / clarify / make excuses for this assessment. But both sides will get a thorough, details, and not-unbiased-but-constently-biased AI assessment of the other … and can use that information to optimize their mutual outcome.

I previously co-founded a startup which did something which kind of rhymed with this; it issued AI-generated reports on software projects’ status, progress, and risks. We moved on because, depressingly, it was clear what the market really wanted was for us to tell companies which engineers to lay off, and that wasn’t something we wanted to devote years of our lives to.

But I think it’s only a matter of time before someone flips that script and begins to realize that the fundamental problem with hiring is information scarcity, and as with all such problems, it can now be automated away. Many people today are worried AI means no place for them in the future. Maybe. But maybe the reality is entirely the opposite; maybe AI can help them find the place where they can do their best work.

Even more remarkable is that these improvements replicate on models small enough to run on your phone. It turns out “Emerging Reasoning with Reinforcement Learning is Both Effective and Efficient” … and can be elicited from any model with more than ~1.5B parameters.

Let’s not forget it was RLHF — Reinforcement Learning from Human Feedback — which made modern AI a viable consumer product in the first place, by clamping down on models so they don’t generate alarming / offensive outputs … for better or worse. (This also limited their creativity and intelligence, and RLHF ‘alignment’ is an excellent example of the question “aligned with who, and which values, exactly?” … but while those are valid criticisms, the reality is there’s no way ChatGPT could have gone public without RLHF.)

Serious People now talk about using reinforcement learning for all manner of other kinds of model post-training and optimization. OpenAI’s Deep Research, “trained using end-to-end reinforcement learning,” is but one early example. Elsewhere, RL teaches robots to do perfect backflips. What’s most intriguing of all is—to my mind, anywasy—this new RL boom is only the first phase of another, larger, weirder one.

But before we get into that…

What exactly is reinforcement learning anyway?

Reinforcement learning is, at heart, good old trial & error. Take a neural network; give it some options; have it choose one; tell it whether its choice was good or bad; tweak its weights to increase its chance of choosing the good option; repeat a few billion times. If your training algorithm is good enough, if a path to the desired outcome exists, and you have the compute, then your model will find that path … or perhaps another path, one you never expected, desired, or even understand.

This is quite different from other forms of machine learning:

• Supervised (learning patterns in datasets painstakingly created/labeled by humans)

• unsupervised (recognizing patterns and clusters inherent in data)

• and self-supervised (a kind of hybrid)

are all, at the end of the day, ways to teach models how to classify input data. Reinforcement learning teaches them how to act.

The RL optimization process is sometimes called hill-climbing; the model’s objective is the top of the mountain, and the crudest version simply keeps climbing the steepest slope it can find. (This may sound like gradient descent, used in backpropagation; it’s not dissimilar, but the important distinction is that you don’t have to mathematically calculate a gradient, you just need be able to score the possible actions.) If your RL algorithm is sufficiently subtle and thoughtful, your model will climb as relentlessly as Sisyphus, overcoming countless false summits, discovering hidden crevasses that lead to secret new heights, until it finally achieves the top of its world.

Sounds amazing, doesn’t it? But everything is harder than it sounds. Until recently RL (as reinforcement learning is universally known) was kinda 2010s. DeepMind used it for AlphaGo, and abandoned it in favor of the transformer. OpenAI experimented with it early on, too, and likewise largely rejected it … but in a very interesting way.

Let’s talk about unnatural selection

In April 2017, two months before the now-legendary paper “Attention Is All You Need” introduced the transformer, OpenAI published “Evolution Strategies as a Scalable Alternative to Reinforcement Learning,” in which:

We explore the use of Evolution Strategies (ES), a class of black box optimization algorithms, as an alternative to popular RL techniques … we highlight several advantages of ES as a black box optimization technique: it is invariant to action frequency and delayed rewards, [and] tolerant of extremely long horizons.

The great Lilian Weng expands on Evolution Strategies:

… one type of black-box optimization algorithms, born in the family of Evolutionary Algorithms (EA) … Evolutionary algorithms refer to a division of population-based optimization algorithms inspired by natural selection … Exploration (vs exploitation) is an important topic in RL. [A variant of ES] encourages exploration by updating the parameter in the direction to maximize the novelty score.

She goes on to cite the interesting POET paper:

An important question is whether [machine learning] problems themselves can be generated by the algorithm at the same time as they are being solved. Such a process would in effect build its own diverse and expanding curricula, and the solutions to problems at various stages would become stepping stones towards solving even more challenging problems later in the process. The Paired Open-Ended Trailblazer (POET) algorithm introduced in this paper does just that…

Me, whenever I see the phrase evolutionary algorithm, my ears prick up.

The legendary-if-only-to-me Thompson tone generator

Long ago, when I was a Waterloo undergraduate, a professor took ten minutes to tell us an engineering anecdote which has stuck with me ever since. Specifically, the tale of the Thompson tone detector:

Dr. Thompson dabbled with computer circuits in order to determine whether survival-of-the-fittest principles might provide hints for improved microchip designs … he employed a chip only ten cells wide and ten cells across⁠— a mere 100 logic gates … cooked up a batch of primordial data-soup by generating fifty random blobs of ones and zeros … For the first hundred generations or so, there were few indications that the circuit-spawn were any improvement over their random-blob ancestors. But soon the chip began to show some encouraging twitches … Finally, after just over 4,000 generations, the system settled upon the best program … He pushed the chip even farther by requiring it to react to vocal “stop” and “go” commands, a task it met with a few hundred more generations of evolution.

Vaguely interesting, right? But now we get to the good stuff:

As predicted, natural selection had successfully produce specialized circuits using a fraction of the resources a human would have required. And no one had the foggiest notion how it worked. The plucky chip was utilizing only thirty-seven of its one hundred logic gates, and most of them were arranged in a curious collection of feedback loops. Five individual logic cells were functionally disconnected from the rest⁠— with no pathways that would allow them to influence the output⁠— yet when the researcher disabled any one of them the chip lost its ability.

The five separate logic cells were clearly crucial to the chip’s operation, but they were interacting with the main circuitry through some unorthodox method⁠— most likely via the subtle magnetic fields that are created when electrons flow through circuitry, an effect known as magnetic flux. There was also evidence that the circuit was not relying solely on the transistors’ absolute ON and OFF positions like a typical chip; it was capitalizing upon analogue shades of gray along with the digital black and white.

That Thompson tone detector is is the closest thing to outright magic that modern engineering has ever discovered … at least until modern AI, with which it shares its frustrating inconsistency, lack of interpretability, and slightly eldritch vibe. As the product of an evolutionary algorithm ourselves (you know, the one called “evolution”) it behooves us to have considerable respect for the power of the approach … along with concern for its baffling incomprehensibility.

Can we use not just RL, but evolutionary algorithms, to improve LLMs? We can say with cautious but increasing optimism that the answer is yes. To the above papers I add this interesting recent Substack post, “How I came in first on ARC-AGI-Pub using Sonnet 3.5 with Evolutionary Test-time Compute,” by Jeremy Berman:

I was inspired by genetic algorithms and have been referring to this approach as Evolutionary Test-time Compute … LLMs can compensate for their generalization limitations through scaled test-time compute guided by evolutionary principles. By generating waves of candidate solutions and using their performance to guide further compute allocation, we can efficiently explore the solution space even with limited prior knowledge.

So what exactly are you suggesting?

I don’t pretend to be an AI researcher; just a (not-so-) humble AI engineer. That said, it seems very likely to me this reinforcement learning renaissance will presage a profusion of new attempts to incorporate evolution strategies—and evolutionary algorithms more generally—into LLMs, including both training and inference.

Speculating even more wildly; is this, perhaps, an aspect of what SSI is doing? Maybe! Ilya Sutskever was a co-author of that Evolution Strategies OpenAI paper. Though I wouldn’t be surprised if they were working on evolutionary algorithms, and recurrent neural networks, and GANs, all at the same time. That fireball called “transformers” has (understandably) consumed almost all the oxygen in the field over the last five years; but as and when they begin to S-curve, there are plenty of other approaches to be added to the arsenal.

Put another way: tasks which currently seem Sisyphean may instead become Icarean.

Evolutionary irreducibility

One quick coda, since I’m already speculating wildly about my pet obsessions. I can’t talk about evolutionary algorithms and not talk about cellular automata. I mean the following is word-for-word from my latest novel:

Conway's Game of Life, devised by mathematician John Conway on graph paper in 1970. In the game, as time ticked forward, the “life” or “death” of a square depended on very simple rules: live squares with two or three live neighbors survived; dead squares with three live neighbors came to life; and all others died. From these deceptively simple constraints, extraordinary complex systems emerged.

The more general term for such games, or systems, was “cellular automata.” They had fascinated mathematicians, engineers, and philosophers for generations with their ability to turn simple rules and patterns into astonishing complexity and chaos. It had been shown that the Game of Life, like most cellular automata, was itself a kind of programming language that could incorporate into itself anything expressible in software. Some avant-garde scientists such as Stephen Wolfram had even suggested, long before [spoiler elided], that our universe itself was a form of cellular automaton.”

At this point it is customary to start talking in a slightly wild-eyed manner about how the fundamental substrate of the universe might be more like software than hardware, and about how evolutionary algorithms might unlock the secrets of the universe. Sure, maybe. But actually I want to issue a related caveat: the study of cellular automata helps to show that there are hard limits to what we can do with software alone, no matter what magic we throw at it.

That hard limit is called computational irreducibility. As Wolfram puts it in “Can AI Solve Science?”, a piece which very explicitly reinforces Betteridge’s Law:

To this ultimate question we’re going to see that the answer is inevitably and firmly no … the point is that if we want to “solve everything” we’ll inevitably be confronted with computational irreducibility, and there just won’t be any way—with AI or otherwise—to shortcut just simulating the system step by step … discrete computational systems like cellular automata are rife with computational irreducibility.

Cellular automata are another form of evolutionary information, but one which show the limits of what we can do with AI, or any other form of computation; no matter how intelligent any entity is, there are hard limits to what it can do. If you ever feel like Sisyphus in an incomprehensible world, there is perhaps some slim consolation in knowing any future superintelligences will likely still feel the same.

I expected it to be divisive. I was not disappointed. It certainly has both the most and the most entertaining one-star reviews of all my books:

Today it is out in paperback in fine bookstores across Canada and the USA. Those sheaves of paper now sitting on shelves had a very long gestation, even by the slow-motion standards of publishing. I started writing Exadelic more than five years ago, in late 2019, when I was directing the GitHub Archive Program, storing the world’s open-source software on 1,000-year archival film beneath an arctic mountain in Svalbard, as well as with various archival partners (the Bodleian, the Library of Alexandria, Stanford Libraries, Software Heritage, the Internet Archive…)

As such, part of my job was to seriously think about what the state of humanity might be a thousand years from now. Meanwhile, GPT-2 had just dropped … and OpenAI had decided it was too dangerous to be released into the world. In retrospect that was completely ridiculous; but it was an excellent science fiction seed.

We live in science fiction. Its power seems inescapable

As was, alas, COVID-19. I wrote the first draft of Exadelic’s crucial fifth act during two weeks of strict quarantine in a hotel room in Canada from which I was not allowed to emerge. Pretty terrible for almost every person in every situation! …But pretty great, in its deeply annoying way, for a writer.

Ironically, three years later, on the day the hardcover finally dropped, I had COVID again, acquired at an off-site for the company which employed me, Metaculus, a platform for forecasting the future. I have since moved on to FutureSearch, which uses AI to research and explore the future of AI.

All of which is to say: we already live in a science fictional world. Thousand-year software archives, global pandemics, AI too dangerous to release, technologies for rigorously predicting the future … it is really not so long ago that these would have only been the stuff of science fiction. Now they are quotidian life.

Maybe this is why so much science fiction now seems so strangely dull and lifeless. It’s not easy for any strangenesses put on a page to even match, much less exceed, the futuristic weirdness in which we all now live immersed…

…but I’m very very confident Exadelic succeeded at that goal, at least. I refer you once more to the abovementioned one-star review, and quite a few more.

Maybe if I’d named him Hiro Protagonist

I’ve written about how much I enjoyed breaking the rules of the form and the Gnostic roots of the simulation hypothesis elsewhere, so I won’t belabor those here. I will say that I thought, and still think, it has a reasonable chance of becoming a cult book … but I now have amusingly mixed feelings about that prospect. I knew going in that a first-person tale of a demographically similar protagonist would lead to many readers conflating said protagonist and me. But in retrospect, if I had known just how much that would happen, I might not have made him quite so thoroughly mediocre, hapless, confused, and consistently wrong about everything…

(OK, yes, I still would have, as one of the many things the book is doing is riffing on the Chosen One trope by turning it into a fifth-act punch line. But maybe I would have been a little less subtle about it, and about a few other things. Mike Ford, who I’m sad I never met, once said he had a “horror of being obvious,” which I share; …and perhaps as a direct result, I’m still not sure anybody else has noticed the sly little joke on the very first page of Exadelic, which admittedly is only visible on a reread.)

I kid, I kid. I’m proud of all my books, but I’m extremely proud of Exadelic, and especially of how weird and divisive it is — please don’t tell my agent, as this is the exact opposite of a path to commercial success — and I will be very happy to see it as a paperback, the form factor in which I consumed all the weird and seminal science fiction of my own youth.

It is a rollicking, fast-paced, funny adventure novel described, with unexpected near-unanimity, as a ‘wild ride.’ It is also, in places, very dark. All my books have dark places. (Heck, the first one was titled Dark Places.) I am not a horror writer but all my books have elements of it. More importantly, though, Exadelic is meant as an exploration of what it means to be human, and any such exploration really has to include both our darker urges and our capacity for suffering … just as any story which deals with the creation and destruction of entire worlds has a moral obligation to zoom in and show the visceral implications for any pawns in such great games.

o3 won’t be personally offended, right?

I leave you with a little AI analysis of my AI novel. For fun I dumped its Goodreads reviews into ChatGPT (o3-mini) and asked what both fans and critics found in common. Editing down the slop:

The reviews paint a picture of a book that is as polarizing as it is ambitious—a wild, idea-packed ride that splits readers into two camps.

Commonalities Between Both Groups

• High Density of Ideas: Regardless of whether they liked it or not, nearly every reviewer noted the sheer audacity of the author’s vision. The book throws together a staggering array of ideas and influences—Silicon Valley tech culture, dystopian conspiracies, magic, and multiverse theories—in a way that is rarely seen.

• Relentless, Fast-Paced Storytelling: Both camps mentioned the breakneck pacing. The story is structured like a roller coaster, delivering one twist after another while shifting gears unexpectedly from one subgenre to another. Whether it’s a sudden time jump, the introduction of cults and witchcraft, or nods to pop-culture phenomena like Bitcoin and even historical figures, the narrative keeps readers guessing.

I think you’re missing a bunch of the more ostentatiously non-obvious stuff, o3-mini, but all the same, you know what, I’ll take it.

Welp. As it turns out, we hadn’t seen nothin’ yet. Since then—these last two weeks in particular—that story has grown darker and stranger yet, to include multiple murders across America. Yes, really. As probably the first (quasi-) journalist to have written about the group in question in some detail, I feel a certain duty to keep the tale up to date. If you don’t want to read about some of the more unhinged fringes of the human experience, I understand; but if you care to know more about the Zizians, read on.

Extropia’s Children was mostly an online oral history, and this too will be a quick overview of existing sources. Other summaries exist; good ones are here and here.

The Backstory

As previously described, the Zizians were one of several groups amid the Bay Area rationalism scene of the 2010s who exhibited cult-like behavior and/or psychotic divergence from consensus reality. In November 2019 they were arrested in Sonoma County, north of San Francisco, while protesting “artificial intelligence, the Center for Applied Rationality, and the Machine Intelligence Research Institute.”

The Zizians are named after their leader Ziz, who is, at least according to a web site built to warn people about her, “particularly interested in AI risk researchers,” “seems to go out of her way to target transgender people” (Ziz is a trans woman herself), and “preaches a doctrine of radical high-tech veganism that promotes punishing your ideological enemies.” Ziz is also, remarkably, a hemispherist:

Ziz writes that each person has two cores made up of their left and right hemispheres. Each of these is considered a full person, and when Ziz says "person" she really means an animal's core.

As of the writing of Extropia’s Children, Ziz was thought to have contributed to the suicide of a woman known as Maia, and the Zizians were clearly, as I put it then, “deeply sad and completely batshit.” Even that was an understatement. Only a few months later they began to leave multiple murders and murder attempts in their wake.

The First Murders

In February 2023 one ‘SefaShapiro’ posted to Medium “A community alert about Ziz.” To tl;dr its tl;dr:

• In August 2022 Ziz seems to have faked her own death in a boating accident. A close associate Gwen may have also faked their death in April 2022.

• In November 2022, three associates of Ziz attacked their landlord in Vallejo, California. One stabbed him with a sword; he shot two of them, killing one; the survivors were arrested. Ziz and Gwen were seen by police at the scene, alive. (The landlord apparently “miraculously survived being stabbed multiple times, had a sword impaled through his chest, and ultimately lost his right eye.”)

• The parents of another associate, Jamie Zajko, were murdered in their home in Pennsylvania on (most likely) the last day of 2022. It is thought plausible that Ziz was involved.

• Ziz was taken into custody for obstructing justice, but in June 2023 her bail was lowered and she was released.

SefaShapiro goes on to mention “Some members of the group apparently spent time together in Orleans County, Vermont before the murders.” This is suddenly highly relevant, because—

January 2025

“A 21-year-old woman was arrested and charged on Friday after the shooting death on Monday of a U.S. Border Patrol agent near the Canadian border,” quoth The New York Times. The shooting happened on January 20. The shooter was one Teresa Youngblut, all of 21 years old, a University of Washington computer science student.

Her companion—killed by the Border Patrol in the resulting shootout—was (like me…) a University of Waterloo alum, and (not like me…) an International Math Olympiad gold medal winner and former Jane Street intern, impressive credentials of unusual intelligence. That said, their reported actions were bizarre and dumb: “skulking about smalltown Vermont dressed in black tactical gear” and openly carrying firearms (legal in Vermont, but suspicion-inducing) followed by fleeing their hotel shortly after police stopped them to inquire what they were up to.

What were they up to? Why did they open fire on the Border Patrol? We still have no idea. We do know that the FBI found “guns and ammunition, as well as an odd assortment of tactical and computer gear” in their car.

These deaths were quickly connected to the Zizians by none other than Jessica Taylor (who herself appears in two different sections of Extropia’s Children) on Twitter, and knew Youngblut’s deceased companion:

The Vermont prosecutor quickly made the same connection:

To quote The National Post quoting the prosecutor’s memo: “Both the defendant and the person who purchased the firearms in Vermont … are acquainted with and have been in frequent contact with an individual who was detained by the Commonwealth of Pennsylvania during that homicide investigation; that individual is also a person of interest in a homicide investigation in Vallejo, California.”

To which you might think:

1. Who could that person possibly be?

2. Murders in faraway Vermont and Vallejo in the same week?

Yep. Curtis Lind, the Vallejo landlord almost murdered by the Zizians in November 2022, was stabbed to death on January 17th of this year, three days before the Border Patrol shooting. One week later, one Maximilian Snyder, who apparently studied computer science and philosophy at Oxford, was arrested in Redding for the crime. And guess what? “In November, Snyder appears to have taken out a marriage license in Washington state’s King County with Teresa Youngblut.”

You won’t find a clearer connection between two distant brutal homicides than that. Apparently Lind’s killing wasn’t simply revenge: court documents suggest he was “killed for the purpose of preventing his testimony in a criminal proceeding.”

Where We’re At

As of January 29, much remains unknown:

• where Ziz and Jamie Zajko are

• how many Zizians there are, and where they are

• what they were trying to do in Vermont,

• what kind of testimony from Lind they were trying to forestall.

This is now getting a fair amount of mainstream media play … but few go back to where this all began, in the Bay Area’s rationalism / AI safety community in the 2010s. It’s admittedly a pretty tortuous history. As a friend of mine put it, “Man do I feel for the public defender.”

Regardless of what comes out in court, this already sounds like one of the most bizarre and bloody murder cults in California history, which is saying a lot. I find myself going all the way back to the Manson Family for a comparison. Will we look back and see the Zizians as the thin edge of a wedge of AI madness? I hope not, and I genuinely don’t think so … but I confess I can’t rule it out entirely.

This leaps to mind when considering the looming dread / hype / apotheosis called Artificial General Intelligence. There is a wide consensus that if/when we reach AGI, it will be a watershed (and/or catastrophic) moment for not just modern civilization but the entire history of humanity. More prosaically, hundreds of billions of dollars may ride on it; AGI is, or at least was, the contractual breakpoint at which Microsoft loses access to OpenAI’s models.

But is there a similarly wide consensus about what we’re actually talking about? Do the formal prescriptive definitions match the colloquial descriptive ones? Is everyone just excitedly talking past everyone else?

Everyone says … oh wait, these are quite different actually

Some formal definitions, and/or prominent non-definitions, of AGI:

• “AI with a human level of cognitive function, including the ability to self-teach.” (Investopedia)

• “A machine that possesses the ability to understand or learn any intellectual task that a human being can.” (Google)

• “Software with human-like intelligence and the ability to self-teach.” (Amazon)

• “A form of AI that possesses the ability to understand, learn and apply knowledge across a wide range of tasks and domains.” (Gartner)

• “Highly autonomous systems that outperform humans at most economically valuable work.” (OpenAI charter)

• “AI systems that can generate at least $100 billion in profits.” (OpenAI contract)

• “an imprecise term that has gathered a lot of sci-fi baggage and hype. I prefer ‘powerful AI’ or ‘Expert-Level Science and Engineering’ which get at what I mean without the hype” (Anthropic, or at least Dario Amodei)

• “a single unified software system that can … reliably pass a 2-hour, adversarial Turing test, assemble a 1:8 scale model of a Ferrari, and pass [a bunch of Q&A tests and interview problems] with a 90% average.” (Metaculus)

• “We introduce six principles for a clear, operationalizable definition of AGI … with these principles in mind, we introduce our Levels of AGI ontology … Our ‘Competent AGI’ level is probably the best catch-all for many existing definitions of AGI … at least 50th percentile of skilled adults across a wide range of non-physical tasks, including metacognitive tasks like learning new skills.” (DeepMind)

It’s reasonable to look at this list and have these reactions, in order:

1. “oh, those are all pretty similar”

2. “wait, they’re actually quite different”

3. “wow, DeepMind wrote an entire academic paper to get to pretty much the same definition Investopedia-of-all-places tossed off in fifteen words.”

Riders of the Jagged Frontier

Some of the discontinuities are obvious. Gartner wants the AI to understand its knowledge. OpenAI, and only OpenAI, measures AI by economically valuable work (which has the huge advantage of being relatively tangible/measurable) while uniquely requiring it to be highly autonomous. DeepMind says 50th percentile of human ability while Amodei says Expert-Level. To us, at least, that’s a big difference!

But more concerning yet are all the carefully vague phrases: human-like, skilled adults, wide range. How human exactly? How skilled exactly? How wide exactly?

It’s well accepted that AI advances along a ‘jagged frontier,’ i.e. it gets very good at some things we associate with intelligence (e.g. LLMs writing code) with astonishing rapidity, while remaining hilariously bad at others (e.g. LLMs guessing how many ‘r’s are in the word ‘strawberry’) with remarkable persistence.

I’ve argued before that we are “prone to a Fundamental Extrapolation Error, in that we extrapolate assuming modern AI is on a path to intelligence like ours.” In other words, people look at the jagged frontier and conclude its valleys mean AI is ‘really’ dumb/fake/fraudulent, or, conversely, its peaks mean the things it’s bad at will be fixed Any Day Now. Terms like human-like or skilled adults only exacerbate this tendency.

What if our g is not their g?

It may be true that in humans most cognitive abilities are fairly well correlated, and when we talk about intelligence we are ultimately usually talking about a single-dimensional variable, often called g. But even if we stipulate for the sake of argument that this is true for human beings … it does not mean it’s true for all beings.

Suppose g was very well correlated with throwing a ball with great precision. Makes some intuitive sense, right? Hand-eye coordination is mostly in the mind! That’s a lot of complex calculus in a split second, especially for e.g. a curve ball! If that were the case, Einstein would also have been a superstar baseball pitcher, and we’d talk about how throwing precision is obviously a member of the “intelligence cluster” of human traits which also includes memory, pattern-matching, ‘advanced’ mathematics and language and music theory, etc. But we don’t. Is it foreordained as a law of the universe that we don’t?

It’s true that Moravec’s Paradox might suggest so … but that so-called paradox is itself a far more jagged frontier than we may appreciate, in an age when full-size humanoid robots can do perfect backflips.

Ultimately, if you partition human intelligence into N separate ‘peaks’ — the Jagged Range that AI researchers are attempting to scale — then on some K of those peaks, modern AI (especially LLMs) can or will soon exceed human capacity, and on N-K they can’t or will not soon. Must K equal N for us to achieve AGI? Seems unlikely. So the question “what is AGI?” if measured purely by comparison to human performance across a wide range of tasks, is simply: what fraction of N does K need to be?

That … doesn’t sound very satisfying or useful, does it?

AGC, or, if you’re so smart, why ain’t you doing something?

This dissatisfaction is, in part, because AGI—descriptively, in common parlance, when we talk about it casually—is only partly about intelligence. It is also largely about capabilities. AGI is in large part really AGC.

Intelligence is important to any given capability, but rarely either necessary or sufficient. For sufficiency, consider the many child prodigies who go nowhere; consider the need for I/O and/or physical limbs; consider a person who is 10x smarter but 1000x slower at a given cognitive task. For necessariness, remember that many skills were deemed cognitive ones, requiring intelligence—such as mathematicaly calculations, or rapid sorting—until traditional computers existed.

A great deal of ‘AGI’ confusion stems from vagueness about whether we are talking about process or outcome. This is the appeal of the OpenAI definitions and the Amodei non-definition. They focus on capabilities and outcomes, not intelligence and process. They do not require AI’s particular set of cognitive strengths and weaknesses to closely match our own in order to be considered sufficiently ‘general.’

Never taxonomize when you can simply ride piggyback

What’s more, plenty of taxonomies of human outputs — or at least occupations — already exist. On Wikipedia; at the Bureau of Labor Statistics; on www.gov.uk and elsewhere. Now, it’s true that comparing an individual human job to an AI capability is apples to pomegranates. Individual jobs are implicitly human-shaped, and assume baseline human limitations and capabilities, whereas the territory of AI requirements, limitations, and possibilities is enormously different.

But you can, in principle, measure the ROI of both. Consider one of the most handwavey of all jobs; that of an architect. We cannot simply imagine an architect’s job shared between a human and an AI on alternate days. Humans are proactive and self-guided, go into offices and to conferences and socialize with fellow architects, work forty hours a week for a fixed salary, etc. The entire concept of an architect’s job has to be entirely broken down, restructured, and reframed to even consider having an AI do even part of it.

…But we don’t actually have to care about that to define AGI. We merely have to measure the inputs and outputs in strict economic terms. If you pay a human architect $100,000 a year, and you also construct an AI-architect system and pour $100,000 of compute into over a year, how comparable are the outputs?

That is, again at least in principle, an answerable question. How do you quantify an architect’s output? Conveniently, economists try to quantify such things all the time; it is a cornerstone of the profession. Here are a few examples. If we can calculate the return—call it the “return on salary-equivalent tokens,” for LLMs—for architecture, we can certainly calculate it for more concrete fields like software.

What I talk about when I talk about AGI

And that, at long last, brings us to my definition of AGI. It is boring. It is quantitative. It focuses on outcomes and capabilities, not processes and intelligence. And that is as it should be, because it has become very clear that if you give human beings an excuse to think in a woolly, fuzzy, aspirational way about artificial intelligence, they will latch onto it with both hands and both feet as well. To wit:

AGI is when 2/3 of a representative sample of professional economists agree that the return on AI inference investment is within 25% of the average return on the equivalent average human salary for at least 50% of occupations, as taxonomized by the US Bureau of Labor Statistics, for which interaction with the physical world is not essential.

(We can drop that last clause when we get AI-driven humanoid robots…)

Do I expect the world to adopt this definition? I sure do not. But I do think it is a useful lens through which to view other definitions of AGI, as the hype / scorn / dismay / excitement spirals up to ever greater and more dizzying heights.

The year is …

wait for it …

1983.

The political parallels are amusing … but the technological parallels are genuinely remarkable. Just as in the early days of computing, we have a behemoth that invented much of the basic technology, but has fallen behind in its sclerosis; an aggressive upstart seizing hearts and minds worldwide in a whole new way; and a smaller, more stylish upstart that seems especially beloved by tech hipsters and artisans.

Back then we had the operating system wars. Now we have the chat wars:

• ChatGPT is Microsoft: the best-known, the people’s choice, the first in a critical new category, the de facto standard.

• Claude is Apple: purists will tell you it’s definitively better, and in many cases they’re right, and it’s the overall choice of the educated classes, but nowhere near as successful when it comes to the masses.

• Gemini is IBM: they invented half this stuff, but left it to others to actually implement and ship it, and are now scrambling to catch up. (Remember OS/2?)

• Mistral’s Le Chat is Sinclair: sorry.

• Llama is … and I know this sounds weird, it absolutely is … GNU/Linux. Right?! OK, Linux didn’t exist in 1983, but the GNU project began that year. And, I mean, the parallels are strong; open software that you can happily download and tinker with and tweak on your own computer! …It just feels odd that this time it’s being provided by a trillion-dollar company rather than a few MIT weirdos.

Note that I talked about the interfaces - ChatGPT, Claude, Gemini - rather than the models. The interfaces are what ordinary people use (or never use, in the case of Llama and Linux.) The models themselves map much more closely to the microprocessors of the 1980s. As such:

• OpenAI is Intel: complex, sometimes needlessly and bafflingly so (Assistants API and Code Interpreter, anyone?) for historical reasons. Lots of options.

• Anthropic is Motorola: they “led the wave of technologies that spurred the computing revolution in 1984, powering devices from companies such as Apple, Commodore, Atari, Sun, and Hewlett-Packard” … but they were no Intel.

• Google is—again—IBM: just as Google’s Vertex AI will serve Claude, did you know that IBM teamed up with Apple and Motorola to build their own microprocessors, the PowerPCs? It did not go well, although they are active on the Mars rovers as we speak.

• Llama is … uh. OK, these analogies, otherwise so compelling, kind of fall down here. There was no open source microprocessor, because microprocessors were/are insanely expensive to make. As are frontier models! … and yet Meta has, in its infinite generosity (hmmmmm….) bequeathed a semi-open one to us.

Looking at the above, you might conclude that the future is bright for OpenAI and Anthropic, dark for Google and Mistral, and unknowable for Meta. However. Google has something IBM didn’t; the example of IBM. (Which to be clear is still a very large and profitable company! …but does not dominate the tech industry as it did in 1983.)

It’s true Google published “Attention Is All You Need” and then apparently did nothing with it for a long time. But they also acquired DeepMind ten years ago, which could yet go down as the most prescient acquisition of all time. They’re trying to move as nimbly as they can. They have acquihired amazing talent for amazing prices. NotebookLLM was a bottoms-up hit. They still have a real chance of continuing to dominate! If only IBM had had their past selves as an example, too…

You’ll note I didn’t mention DeepSeek or any other Chinese models. That’s because, like Meta, there simply was no comparison back then; technology was an American business. That aspect of the future, too, seems impressively unknowable. Time isn’t entirely a flat circle. But it sure does rhyme sometimes.