Prompt engineering for production: structure, schemas and version control
A prompt that works in a playground and a prompt that works across ten thousand varied real inputs are different artefacts. The gap between them is structure, explicit failure handling, and treating the prompt as versioned code.
By Quality AboveAll · · 8 min read
- Specify the output schema explicitly and validate against it, because a prompt that usually returns valid structure will break your parser eventually.
- Define what the model should do when it cannot answer, or it will invent something.
- Version prompts in your repository and run them against a regression set, exactly as you would any other behaviour-defining code.
Structure beats eloquence
Effective production prompts are organised, not clever. Separate the role, the task, the constraints, the input data and the required output format into distinct labelled sections. Models follow structured instructions far more reliably than prose paragraphs, and structured prompts are enormously easier for your own team to modify safely.
Put the instructions before the data when the input is long, and restate the critical constraint after it. Attention to material in the middle of a very long prompt is measurably weaker, so the position of an instruction affects whether it is followed.
Specify the output contract
Anything a downstream system will parse must have an explicitly specified shape, ideally with a schema and an example of a valid response. Then validate every response against that schema and handle failures deliberately, with a retry or a fallback, rather than assuming compliance.
Use the structured-output or tool-calling features your provider offers where available, because a constrained decoder is far more reliable than an instruction politely requesting valid output. This single change removes most of the flakiness teams attribute to model quality.
Never let an unvalidated model response reach a system that assumes it is well formed. Somewhere in the next ten thousand calls, it will not be.
Tell it what to do when it cannot answer
The most consequential line in most production prompts is the one defining the refusal path: if the provided context does not contain the answer, say so and stop. Without it, models fill gaps, which is precisely the behaviour that erodes trust in an AI feature.
Pair that instruction with an application-level check on the response, since instructions reduce but do not eliminate the behaviour. Our note on reducing hallucinations covers the layered approach, and AI guardrails covers enforcement outside the prompt.
Treat prompts as versioned code
Prompts define behaviour, so they belong in the repository under review, not in a database field someone edits in production. Every change should be reviewable, attributable and revertible, because a one-word edit can measurably change output across every request.
Attach a regression set to each significant prompt, run it in CI on change, and compare against recorded expectations. This is ordinary engineering discipline applied to a component that happens to be written in English, and it catches the quiet regressions that manual spot-checking misses. Our CI/CD testing guide covers the pipeline mechanics.
Techniques worth the tokens, and ones that are not
Few-shot examples are consistently the highest-return technique, particularly for format adherence, and three well-chosen examples usually beat ten mediocre ones. Asking the model to reason step by step helps on genuinely multi-step problems and mostly wastes tokens on simple extraction.
Elaborate personas, flattery and threats do not reliably improve output and make prompts harder to maintain. Measure any technique against your evaluation set before adopting it, because prompt folklore spreads much faster than prompt evidence.
Frequently asked questions
Should prompts live in the database so non-engineers can edit them?
Only with the same review and testing rigour as code. Prompts define system behaviour, and an unreviewed production edit is a deployment without a pipeline.
How long should a prompt be?
As long as needed to remove ambiguity and no longer. Every token costs money and adds distance between instructions, so verbose prompts can perform worse as well as cost more.
Do prompts transfer between models?
Partially. Structure and examples usually carry over, but adherence and formatting behaviour differ enough that you should re-run your evaluation set after any model change.
Prompts behaving inconsistently once real users arrived? A free technical review will show you where structure, validation and regression coverage are missing.
