Structuring a team for AI products: the roles that matter and the ones you can skip
Most organisations do not need to hire a research team. They need two or three good engineers, one clear owner, and access to whoever actually does the work being augmented.
By Quality AboveAll · · 8 min read
- The most commonly missing role is an owner who reviews outputs regularly and decides what good enough means.
- Strong application engineers plus a domain expert outperform a siloed data science team for most product AI.
- Embed AI work in the team that owns the product, rather than creating a centre of excellence nobody can access.
The roles you genuinely need first
An engineer comfortable with APIs, retrieval and probabilistic behaviour can build most of what a first AI feature requires. A second engineer for the surrounding integration and interface work usually doubles throughput because those halves proceed in parallel.
The critical third role is the owner: someone accountable for whether the feature is useful, who looks at real outputs weekly and makes the judgement calls. Without that person, the team optimises for whatever is measurable rather than what matters.
The domain expert is not optional
Whoever currently performs the task being augmented must be closely involved. They know the exceptions, the unwritten rules and what a wrong answer actually costs, and none of that is discoverable from the documentation.
They are also the people who label your golden dataset and review early outputs. Treating them as stakeholders to be consulted quarterly, rather than participants in the weekly loop, is the most reliable way to build something technically sound and practically useless.
The person who does the job today is not a stakeholder to be surveyed. They are the specification.
When you need specialists
A dedicated machine learning specialist is warranted for custom model training, non-trivial fine-tuning, or genuinely novel problems. For retrieval and generation over your own content, strong application engineers with good evaluation discipline are generally more effective.
MLOps expertise becomes worthwhile once you run models yourself or operate several AI features in production. Before that, standard platform engineering practice, covered in platform engineering, is the right foundation.
Where the team should sit
Embedded in the product team beats a central AI group for anything user-facing. Centralised teams accumulate a queue, lose product context, and become a dependency rather than a capability.
A small central group is useful for shared infrastructure, evaluation tooling, model gateways and governance, and for spreading practice. The distinction is that it enables product teams rather than doing the work for them.
Building the capability across the organisation
The teams that progress fastest are the ones where many engineers understand what these systems can and cannot do, rather than where AI is one specialist's job. That understanding spreads through practice, not presentations.
Rotating engineers through AI work, reviewing outputs together, and keeping the evaluation suite visible are the mechanics that do it. Our notes on getting a team AI-enabled and AI coaching cover the practical programme.
Frequently asked questions
Do we need to hire a data scientist?
Not for most product AI built on existing models. You need it for custom training, complex fine-tuning or novel research problems.
Should AI sit in a central team or in product teams?
Embed for product features, centralise for shared infrastructure and governance. Fully centralised delivery teams tend to become bottlenecks.
How many people does a first AI feature need?
Two engineers, an engaged owner and access to a domain expert is a workable shape for a first feature. Scale follows evidence, not ambition.
Working out who you need before starting? A free 30-minute consultation covers team shape, and whether you need to hire at all.
