“Today’s frontier LLM research is not about building animals. It is about summoning ghosts.”
These are the words of Andrej Karpathy—co-founder of OpenAI, former head of AI at Tesla, and now working with Anthropic’s pre-training team. To understand what he means, it helps to know the fault line running through AI research—what Karen Hao, in Empire of AI, calls the split between the symbolists and the connectionists.
The symbolists argue that large language models are sophisticated statistical systems (stochastic parrots) that generate probable language without true understanding. The connectionists believe scaling neural networks with more data and compute will eventually produce artificial general intelligence (AGI).
Karpathy’s ghost framing doesn’t fully side with either camp. He’s not arguing that LLMs understand nothing, nor that scale will eventually produce something like consciousness. He’s arguing for something more precise. LLMs, he writes, are “muddled by humanity. Thoroughly engineered by it. They are these imperfect replicas, a kind of statistical distillation of humanity’s documents with some sprinkle on top.”
The animal/ghost distinction is what makes this precise. An animal learns from contact with the world—hunger, fear, memory, survival pressure. A ghost is different. A ghost is a trace that behaves like a being. It speaks and acts, but what it speaks is assembled from what remains.
This is the nature of the LLM itself. But when you connect one of these models to your organization’s content—your product briefs, your knowledge base, your messaging frameworks—you add a second layer with the same problem: traces from the past. The question is: whose traces? And in what condition have they been left?
What the ghost in the machine is allowed to know
The ghost doesn’t know your product. It only knows your product’s language. These are not the same thing.
An AI writing system generates content by retrieving, pattern-matching, and assembling from whatever language it has access to. Consider how three common enterprise platforms handle this:
- Writer builds an explicit knowledge graph at the center of its architecture. Someone has to decide what goes in: which product briefs, which messaging frameworks, which approved terminology. That decision is visible, deliberate, and consequential.
- Microsoft Copilot for M365 uses Microsoft Graph to connect to everything in your Microsoft environment—SharePoint, Teams, OneDrive, Outlook. If the implementation lacks a focused curation step, you may end up including stale positioning decks, superseded product briefs, or campaign copy from strategies that have since changed. All of it could be in scope by default without you even knowing it.
- Jasper, widely used by content and marketing teams, supports knowledge base connectors and brand voice controls, but the governance of what goes in—and what stays out—should depend on editorial choices that many teams might not even make.
Three different architectures. The same underlying decision: what does this system get to speak from?
The problem is that for many organizations these decisions are treated as a technical configuration issue. Someone in operations connects the folders, maps the permissions, and moves on. But the decisions should be more editorial than technical.
The ghost will speak from whatever room you build for it. The question is whether anyone with editorial judgment helped design the room.
The haunted house is not a castle
The poet Emily Dickinson knew a lot about language and ghosts. She says:
One need not be a House—
The Brain has Corridors—surpassing
Material Place—
For enterprise content teams, the Chamber or the House is the platform: Writer, Copilot, Jasper—or whatever. These are the visible, external systems organizations evaluate, compare, and worry about. But the Chamber, Dickinson argues, is beside the point. One need not be a chamber to be haunted.
The Brain—the corridors that surpass material place—is the knowledge graph, the retrieval layer, the accumulated body of organizational language that feeds the system. This is where the real haunting lives, and it surpasses the platform in consequence. The threat isn’t so much at the door as inside the organization’s content brain that no one thought to audit.
Content debt, now at scale
Content debt is the accumulated body of language an organization leaves behind as it moves forward—old messaging never retired, terminology that drifted as the product evolved. Before AI, it sat quietly. Now it speaks.
Let’s say you keep a stale page up in a corner of your website. Before AI, the damage was local. AI changes the exposure—and it isn’t only internal. When you connect an AI writing system to your content, dormant material becomes active. What was ignored becomes generative.
Generative engine optimization (GEO) means your published content is now being retrieved and surfaced by AI-powered search systems you don’t control. An old blog post, a press release from a strategy you’ve since abandoned. All of it is fair game. The AI doesn’t know your current position. It knows what it can find.
If one product page says the platform “automates” a process, another says it “augments” it, and a sales deck says it “replaces manual work”—the AI doesn’t resolve the contradiction. It averages it out. Something that sounds reasonable gets rendered fluent and deployed at scale.
Content teams are not downstream producers
In most organizations, content teams have been treated as last in line. Strategy arrives from above. Product truth is defined elsewhere. The content team turns decisions into copy. In the age of AI writing systems, that model becomes dangerous.
If the machine is going to generate and distribute company language at scale, the people responsible for that language cannot sit at the end of the process. Someone has to decide which product description is authoritative when three versions exist. Someone has to govern what goes into the knowledge graph and what stays out.
These are content decisions—questions about language, accuracy, and truth. The knowledge graph curation decision—what the machine is allowed to reach and what stays out—is one of the most consequential content strategy decisions an enterprise will make in the next several years. At many organizations, it is currently being made by people without the editorial training to recognize what they are deciding.
The language that remains
Emily Dickinson has another poem of relevance to this discussion:
When it is said,
Some say.
I say it just
Begins to live
That day.
The moment words are released, they begin a life of their own—indifferent to the context that gave them meaning, indifferent to whether that context still holds.
The content an organization produces outlasts the thinking behind it. The product brief, the messaging framework, the campaign deck—all are often preserved and retrievable by your AI long after the decisions that shaped them have been revised, reversed, or forgotten. The machine doesn’t know the difference. It knows what it can reach.
At its core, this is a story about content stewardship. The ghost will always speak. The only question is whether it speaks from something you still stand behind.
← Back to Blog