Your prompts aren’t in version control. They should be.

Here’s a situation I’ve seen play out more than once. A founder notices that user engagement has dropped over the last few weeks. The product still works — requests succeed, responses come back — but something feels off. The outputs are slightly worse, slightly less useful. Users are churning a little more. Support tickets have ticked up.

When we dig in, the problem is almost always the same thing: someone tweaked a prompt three or four weeks ago, probably for a good reason, and introduced a subtle regression. Nobody noticed immediately because the outputs still looked plausible. By the time anyone connected the dots, there was no clean record of what changed, when, or why.

The fix requires archaeology. Reading through git blame, trying to reconstruct what the prompt used to say, guessing at what changed. It takes days, sometimes, to get back to a known-good state. And the worst part is that there’s no way to tell which specific outputs were affected or how many users saw degraded quality.

This is a prompt versioning problem. And it’s more common than most founders realize, because prompts don’t look like code. They look like text. So teams treat them like copy — something you tweak, polish, and overwrite — rather than like production software that needs the same discipline as the rest of the system.

Prompts are your most volatile code

Think about how often your prompts change versus how often your database schema changes. Or your API routes. Or your core business logic. Prompts are probably changing an order of magnitude more often than any of those things. Every quality improvement, every edge case you discover, every model update that changes behavior — all of it drives prompt changes.

And yet: database migrations have version numbers, rollback scripts, and apply in a controlled order. API changes go through review and deprecation cycles. Prompts? Usually edited in place, committed in a batch with five other things, and forgotten about until something breaks.

This isn’t about rigor for its own sake. It’s about the fact that prompts have runtime effects. A prompt change is a production deployment. The output your users see changes the moment a new version is in effect. If you can’t trace which prompt produced which output, you can’t debug, can’t measure improvement, and can’t roll back confidently when something goes wrong.

What versioning actually means here

This isn’t just “put your prompts in git.” That’s part of it, but it’s not sufficient on its own. The full picture has three pieces:

1. Prompts as first-class artifacts. Each prompt has a unique identifier and a version number. When you change a prompt, you increment the version — you don’t overwrite. You keep the history. This sounds obvious, but it requires a deliberate decision about where prompts live. If they’re scattered inline across dozens of files, this is hard. If they live in a dedicated registry — even just a folder of versioned files or a simple database table — it becomes natural.

2. Deployment awareness. Your system knows which version of a prompt is active in each environment. Staging might be running v4 while production is still on v3. That configuration is explicit and visible, not implied by whatever’s currently checked into main. This is what makes controlled rollouts possible — you can promote a new version to production deliberately, not just by merging code.

3. Output traceability. Every LLM call logs which prompt version produced it. When you look at a specific response that went wrong, you can immediately see: “this was produced by prompt summarize-v3, on this input, at this time.” When you later ship version 4 and want to know if quality improved, you can compare distributions of outputs across versions rather than just eyeballing a few examples.

The simple pattern for V1

You don’t need a dedicated prompt management platform to get the benefits of this. The pattern scales down to something you can implement in a day.

Store your prompts in a dedicated directory — something like prompts/ at the root of your repo. Each prompt is a file with a name and a version: summarize.v3.txt. That’s it for storage. Your application loads prompts by name and version from this directory, not from hardcoded strings in your business logic.

For deployment awareness, use environment config. A simple env var or config file specifies which version of each prompt is active in production right now: PROMPT_SUMMARIZE_VERSION=3. Changing the active version in production is a config deploy, not a code deploy. Fast, explicit, reversible in seconds.

For traceability: tag every LLM call log entry with the prompt name and version. If you’re already logging your LLM calls (and you should be), this is one extra field. Over time, you accumulate a dataset that lets you slice performance by prompt version — which is exactly what you need when you’re trying to prove that v4 is actually better than v3 before you promote it.

The connection to evals

Prompt versioning and evals are related but different. Evals (covered in an earlier post) are about building a test suite that tells you whether your AI is producing good outputs. Prompt versioning is about the operational layer that makes evals useful over time.

Without versioning, your eval scores are ambient. You know quality is 87% right now, but you can’t tie that number to a specific prompt version. You can’t run your eval suite against prompt v3 and v4 separately to see which one scores better. You can’t look back at production logs and correlate the quality drop three weeks ago with the specific change that caused it.

With versioning, your evals get a temporal axis. You can track quality over time, per prompt, and actually know when you’re making things better versus just changing them. That’s a different level of confidence.

What to avoid

Two failure modes I’ve seen with this:

Over-engineering early. Prompt management platforms exist, and some are genuinely good. But adopting one too early means adding a vendor dependency and operational overhead before you have enough volume to justify it. The simple file-based pattern works fine until you’re managing dozens of prompts and need multi-user editing, approval workflows, or non-engineer access. Cross that bridge when you reach it.

Treating versioning as a substitute for judgment. Having a versioned history doesn’t tell you which version is best — it just gives you the data to answer that question. You still need your evals, your user feedback, your quality signals. Versioning is the plumbing. You still have to run the water.

A word on model updates

There’s one more reason this matters that doesn’t come up until it bites you. Model providers update their models. Sometimes the changes are minor. Sometimes a new version of the same model behaves differently enough that prompts that worked great in the previous version produce noticeably worse outputs in the new one.

When that happens — and it will — you need to know immediately, and you need to know which prompts are affected. Without version tracking, you’re relying on users to tell you something changed. With it, you can catch a quality regression in the first few hours of traffic on the new model, roll back the prompts that are behaving strangely, and fix them against the updated model behavior before most users ever see the problem.

That’s the difference between a 30-minute incident and a two-week mystery. And it’s entirely a function of whether you have the logging infrastructure to see it clearly.