What an AI project actually costs, beyond the API bill
Inference is the cost everyone asks about and rarely the largest one. The budget that survives contact with reality includes evaluation, review labour and the maintenance nobody quotes for.
By Quality AboveAll · · 8 min read
- Engineering, not inference, dominates the first year for most AI features.
- Budget evaluation and monitoring explicitly, or they will be cut and you will lose the ability to tell whether it works.
- Assume an annual re-validation cost when the model you built on is superseded.
Discovery and scoping
A short discovery that produces a defined use case, a measured baseline, a data assessment and a success threshold is the cheapest insurance available on an AI project. It is typically days to a couple of weeks.
Skipping it does not save money, it moves the cost to the point where a half-built feature meets an undefined requirement. That is the most expensive place for ambiguity to surface.
Build cost is mostly ordinary engineering
The AI-specific portion, prompts, retrieval, evaluation, is usually a minority of the work. The majority is integration, permissions, interface, error handling and deployment, which is priced like any other feature.
The variables that move the estimate most are integration complexity and data preparation. A feature reading a clean, well-permissioned corpus is straightforward; one that must reconcile three systems with inconsistent identity is not, and that has nothing to do with AI.
Inference cost, estimated properly
Measure tokens per request from real examples, including the full prompt, retrieved context and expected output, then multiply by realistic volume and current pricing. Output tokens typically cost several times input tokens, so verbose responses are expensive twice over.
The common estimation error is testing on short prompts and deploying with eight retrieved passages attached to every call. That difference can be an order of magnitude, which is why the estimate should come from the real prompt shape. Levers are covered in LLM cost optimization.
The API bill is the cost you can see. The review labour and the annual re-validation are the ones that decide whether the feature is worth keeping.
The operating costs that get forgotten
Human review labour, where the workflow requires it, is a real recurring cost and often the largest after engineering. Evaluation maintenance, monitoring, and incident response all consume time that has to come from somewhere.
Then there is model change. Providers deprecate models, and each transition requires re-running evaluation, adjusting prompts and re-validating. Budgeting for one such cycle a year is realistic and keeps it from arriving as an unfunded surprise.
Where to spend and where not to
Spend on discovery, evaluation and the controls that make the feature safe. Underspending there is what produces features nobody trusts and cannot be improved because nothing is measured.
Do not spend on speculative platform work, elaborate orchestration before the use case is proven, or fine-tuning before prompting and retrieval have been exhausted. The staged approach in a 30-day proof of concept keeps early spend bounded and informative.
Frequently asked questions
Is inference cost usually significant?
At low and moderate volume it is often a minor line item compared with engineering. At high volume it becomes a primary concern and deserves active optimisation.
What is the most underestimated cost?
Ongoing human review and the periodic re-validation when models change. Both are recurring, both are real, and neither appears in an initial build quote.
Can we start small to control cost?
Yes, and you should. A narrow first feature with a measured baseline gives you real numbers to extrapolate from instead of vendor estimates.
Need a realistic number before committing to an AI project? A free 30-minute consultation will get you a scoped estimate including the parts most quotes leave out.
