State of GPT: A Deep Dive into Andrej Karpathy's Vision for the Future of Software

If you’ve ever felt that magical moment of bringing an idea to life with just a few lines of English, you’re part of a revolution. You’re a “vibe coder,” and according to former Tesla AI Director Andrej Karpathy, you’re at the forefront of one of the most fundamental shifts in computing in the last 70 years. In a recent talk, Karpathy laid out a compelling vision for what it means to build and interact with software in the era of AI. For all of us learning to vibe with AI, his insights are not just interesting—they’re a roadmap to the future.

This post will break down the major themes from Karpathy’s talk, from the evolution of software to the rise of “people spirits” on the internet and what it all means for us, the new generation of creators.

The Three Eras of Software: From C++ to English Prompts

For decades, software was a set of explicit instructions typed out in languages like Python or C++. Karpathy calls this Software 1.0. Then, a few years ago, a new paradigm emerged: Software 2.0. This new “software” wasn’t written by humans, but by optimizers. It’s the weights of a neural network, trained on massive datasets to perform tasks like image recognition. Think of GitHub as the home of Software 1.0, while Hugging Face is the repository for Software 2.0.

But now, we’re in a new, even more profound shift: Software 3.0. This is the era of Large Language Models (LLMs), which Karpathy describes as a new kind of programmable computer. And what’s the programming language? Remarkably, it’s English. Your prompts are the new code. This means we now have three distinct paradigms for creating software, and being fluent in all of them is crucial.

LLMs: The New Operating System

So what are these new LLM computers? Karpathy offers a few powerful analogies to help us understand them.

Like a Utility: LLMs are provided by major labs (like OpenAI, Google, Anthropic) that spend massive amounts on capital expenditure to train them, much like a power company builds a grid. We access this “intelligence” through APIs and pay for it, demanding reliability and low latency, just like electricity. When a major LLM goes down, it’s like an “intelligence brownout” for the planet.
Like a Chip Fab: The immense cost and deep, secretive research involved in creating cutting-edge LLMs are similar to the world of semiconductor fabrication.
Most Accurately, Like an Operating System: This is the analogy Karpathy finds most fitting. LLMs are not simple, commoditized utilities; they are increasingly complex software ecosystems. We’re seeing a familiar pattern emerge: a few dominant, closed-source providers (like Windows or macOS) and a growing open-source alternative (with Llama as a potential Linux equivalent).

He argues we’re in the “1960s of operating systems.” Compute is expensive, so it’s centralized in the cloud, and we access it through a time-sharing model, just like in the early days of computing. The personal computing revolution for AI hasn’t happened yet, but we’re seeing the first glimmers.

Understanding the “Psychology” of Our New Tools

To effectively program these new computers, we need to understand their unique “psychology.” Karpathy describes LLMs as “stochastic simulations of people” or “people spirits” trained on the vast expanse of human text on the internet. This gives them a human-like, yet alien, set of traits.

Their Superpowers:

Encyclopedic Knowledge: They have near-perfect recall of the immense data they were trained on, similar to the character in the movie Rain Man.

Their Cognitive Deficits:

Hallucination: They frequently make things up.
Jagged Intelligence: They can be superhuman at some tasks but fail at others in ways no human would.
Anterograde Amnesia: LLMs don’t learn from interactions over time like a human colleague would. Their “weights are fixed,” and their context window (working memory) gets wiped clean. Karpathy recommends watching Memento or 50 First Dates to understand this limitation.
Gullibility: They are susceptible to security risks like prompt injection.

To work with them is to manage a superhuman with a unique set of cognitive challenges.

Building for Partial Autonomy: The “Iron Man Suit”

Given the fallible nature of LLMs, Karpathy is cautious about the hype around fully autonomous agents. Instead, he advocates for building “partial autonomy products” that function more like Iron Man’s suit: a powerful tool that augments the user but can also act on its own when directed.

Successful apps like Cursor for coding and Perplexity for search are early examples. They share key features:

Automated Context Management: The app handles the complex task of feeding the right information to the LLM.
Orchestration: They use multiple LLMs for different tasks (e.g., embedding, chat, applying diffs).
Application-Specific GUI: A graphical user interface is crucial. It allows us as humans to quickly audit the AI’s work. It’s much faster to verify a visual diff than to read a block of text explaining a code change.
The Autonomy Slider: The user can choose the level of autonomy, from simple tab-completion to letting the agent modify an entire repository.

The goal is to make the “generation-verification loop” between the human and AI as fast as possible. This means keeping the AI “on a leash” to avoid getting a 10,000-line diff that’s impossible to review.

Vibe Coding and Building for Agents

Perhaps the most exciting shift for us is that everyone who speaks a natural language is now a programmer. This is what Karpathy’s viral “vibe coding” tweet captured so perfectly. It’s a “gateway drug to software development,” empowering a new generation to build things that were previously out of reach.

Karpathy himself has vibe-coded, building an iOS app without knowing Swift and a web app called MenuGene that generates images for restaurant menu items. He found that the coding itself was the easy part. The real challenge was the “DevOps” – the endless clicking in web interfaces for authentication, payments, and deployment. A computer was giving him, a human, instructions. “Why am I doing this?” he asked. “You do it.”

This leads to his final major point: we need to start building our digital infrastructure for agents. Just as robots.txt instructs web crawlers, we might need an llm.txt to tell an LLM what a domain is about in a simple, readable format. Companies like Vercel and Stripe are already creating documentation specifically for LLMs, replacing instructions like “click here” with curl commands an agent can execute.

Your Turn to Build the Future

We are at a remarkable, unprecedented moment. The tools to build the future are not locked away in corporate labs; they have been “beamed down to our computers” for billions of people to use. It’s our time to step in and program these new machines.

The path forward is to build partial autonomy products, creating Iron Man suits that augment human capability. Over the next decade, we will slowly move the autonomy slider from left to right, from augmentation to agent. It’s going to be a fascinating journey, and as vibe coders, you are the ones who will build it.

Watch the Full Talk

You can watch Andrej Karpathy’s full talk, “State of GPT,” here: https://www.youtube.com/watch?v=LCEmiRjPEtQ

Full Citations

Here are some of the most fascinating insights from Andrej Karpathy’s talk, “State of GPT”:

On the Fundamental Shift in Software: “I think roughly speaking software has not changed much on such a fundamental level for 70 years. And then it’s changed, I think, about twice quite rapidly in the last few years.”
Software 3.0 and English as the New Code: “Neural networks became programmable with large language models… your prompts are now programs that program the LLM. And remarkably, these prompts are written in English.”
LLMs as a Utility and an “Intelligence Brownout”: “When the state-of-the-art LLMs go down, it’s actually kind of like an intelligence brownout in the world. It’s kind of like when the voltage is unreliable in the grid, and the planet just gets dumber the more reliance we have on these models.”
LLMs as Operating Systems: “In my mind, LLMs have very strong kind of analogies to operating systems… We’re kind of like in this 1960s-ish era where LLM compute is still very expensive… and that forces the LLMs to be centralized in the cloud.”
The “Psychology” of LLMs: “The way I like to think about LLMs is that they’re kind of like people spirits… they are stochastic simulations of people.”
LLM Superpowers and Deficits: “LLMs have encyclopedic knowledge and memory… But they also have a bunch of I would say cognitive deficits. So they hallucinate quite a bit… They display jagged intelligence… they also kind of suffer from anterograde amnesia.”
Building Partial Autonomy Products: “It’s less like building flashy demos of autonomous agents and more building partial autonomy products. And these products have custom GUIs and UI/UX… so that the generation verification loop of the human is very, very fast.”
Keeping AI on a Leash: “It is in our interest to make this loop [generation-verification] go as fast as possible… we have to somehow keep the AI on the leash because it gets way too overreactive.”
Everyone is a Programmer: “Not only is there a new type of programming language that allows for autonomy in software but also as I mentioned it’s programmed in English which is this natural interface and suddenly everyone is a programmer because everyone speaks natural language like English.”
Building Digital Infrastructure for Agents: “We need to start building our digital infrastructure for agents… Companies like Vercel and Stripe are already creating documentation specifically for LLMs, replacing instructions like ‘click here’ with curl commands an agent can execute.”