Someone edits the system prompt directly in the OpenAI or Anthropic console to fix a bad response, hits save, and moves on. Two weeks later, output quality on an unrelated task drops 15% and nobody can explain why. Nobody can explain why because nobody can see what changed. The prompt lives in a dashboard, a Notion doc, or a string literal three environments removed from the code review that would have caught it. This is how most teams handle prompts, and it's why most teams eventually ship a quality regression they can't diagnose.
The common wrong approach
Most teams treat the prompt as config, not code. It sits in a database row, an admin panel, or a constant buried in a service file that nobody reviews with the same rigor as application logic. Someone with good intentions — a PM chasing a support ticket, an engineer debugging a one-off — tweaks the wording directly against production. It seems reasonable: prompts are just text, the fix is small, and going through a full PR cycle for a sentence change feels like overkill.
It breaks the moment more than one person touches the prompt, or the moment you need to answer "what changed between the version that worked and the version that doesn't." We've seen this exact failure on Email Triage: a prompt edited to fix false positives on one email category silently broke classification accuracy on three others, and it took four days to find because nobody had a diff to look at — just a vague sense that "it used to work better."
The better approach
Prompts are code. Store them as code. At Dainty, every production prompt lives in the repo as a versioned file — usually a `.md` or `.txt` file with a clear name (`triage_classifier_v4.md`, not `prompt.txt`) — and changes go through the same PR review as everything else. That alone fixes 80% of the problem: you get a diff, a commit message, an author, and a timestamp for free.
The other 20% is evaluation discipline. A prompt change without a way to measure its effect is just a guess with extra steps. We pair every prompt file with a small eval set — 20 to 50 representative inputs with expected outputs or scoring criteria — and run it before merging. On Ghost Writer, this is a script that replays saved customer inputs against both the old and new prompt version and diffs the outputs side by side. It doesn't need to be sophisticated. It needs to exist.
Tag prompt versions the same way you'd tag a model version, because the two are coupled — a prompt tuned for Claude Opus 4.5 doesn't necessarily behave the same on a newer model, and vice versa. We log both together at inference time: {prompt_version: "triage_v4", model: "claude-sonnet-5"}. When a customer reports a bad output, you can reconstruct exactly what generated it, instead of guessing which of six recent changes is responsible.
For teams running frequent prompt iteration, a lightweight prompt management layer (LangSmith, PromptLayer, or a homegrown table with version history) adds rollback and A/B comparison without much overhead. The tool matters less than the habit: nothing reaches production without a version number attached to it.
Where this breaks
For a two-person team shipping a single prompt that rarely changes, full PR review and a formal eval harness is more process than the problem deserves — a versioned file with a changelog comment at the top may be enough. The overhead becomes worth it once you have more than one person editing prompts, more than one environment, or a customer-facing feature where a regression has real cost. Eval sets also decay: if nobody updates them as the product changes, they give false confidence that a prompt still works when the underlying use case has moved on. Treat the eval set as a living artifact, not a one-time checkbox.
Practical next step
This week, find every prompt currently living outside your codebase — in a console, a database, a Slack thread someone copy-pasted from — and move it into a versioned file with a commit history. Don't rewrite the prompts yet. Just get them under version control so the next change produces a diff instead of a mystery. If you want help setting up the eval harness that makes prompt changes safe to ship quickly, start a project and we'll walk through what that looks like for your stack.