Building a recommendation engine: from simple baselines to embeddings
Recommendation projects fail less often on algorithms than on evaluation. It is easy to build something that scores well offline and changes nothing about what customers actually do.
By Quality AboveAll · · 8 min read
- Start with a popularity or co-purchase baseline. It is cheap, surprisingly strong, and it is the bar anything clever must beat.
- Content-based recommendations using embeddings solve cold start for new items, where collaborative filtering has nothing to work with.
- Offline metrics are directional at best. Only a controlled online test tells you whether behaviour changed.
The approaches, briefly
Collaborative filtering uses behaviour: people who interacted with this also interacted with that. It needs no understanding of the items themselves and works remarkably well once you have interaction volume, which is also its limitation.
Content-based filtering compares the items directly, and modern versions do this with embeddings over descriptions, images and attributes, so similarity captures meaning rather than shared tags. Hybrid systems combine both, using content similarity where behavioural data is thin and behaviour where it is rich.
Start with the boring baseline
Most popular in category, frequently bought together, and recently viewed are trivial to implement and capture a large share of the achievable lift. They also give you the comparison point without which no later model can be judged.
Teams that skip this step end up unable to answer whether their neural recommender beats "show the bestsellers", which is an uncomfortable question to face after a quarter of work. Build the baseline, measure it, then try to beat it.
If your sophisticated model cannot beat 'most popular in this category', the problem is not that it needs more layers.
Cold start, the recurring practical problem
New items have no interaction history, so behavioural methods cannot place them, and in catalogues with constant turnover that is a permanent condition rather than a launch problem. Content-based similarity handles it directly, because a new item's description and images exist from day one.
New users are the mirror image. Sensible defaults, category-level popularity, and fast adaptation within the session cover the gap, and a short onboarding preference step is often worth more than any model improvement for first-session relevance.
Evaluating without fooling yourself
Offline evaluation on historical data has a structural bias: it can only reward recommending things users already found, which penalises useful discovery. It is a reasonable filter for obviously bad models and a poor predictor of business impact.
The measurement that counts is a controlled online experiment against your baseline, on a metric that matters commercially, and run long enough to survive novelty effects. Watch for the classic trap of optimising click-through while conversion or satisfaction quietly declines.
Business rules and the presentation layer
Pure relevance is rarely the whole objective. Margin, stock availability, contractual promotions and diversity all need to influence the final list, and applying them as a re-ranking layer over model output keeps the model honest and the rules auditable.
Where and how recommendations appear frequently matters more than the ranking itself. The same list performs very differently depending on placement, count and labelling, which is worth testing before assuming the model is the limiting factor. Sector context is in AI in e-commerce.
Frequently asked questions
How much data do I need for recommendations?
Content-based approaches work from your catalogue alone. Collaborative filtering needs meaningful interaction volume, so most products start content-based and blend in behaviour as it accumulates.
Should I build or buy?
Buy when your catalogue and objectives are conventional and a hosted service fits. Build when your ranking depends on business rules or data a generic service cannot see, using the reasoning in our build versus buy framework.
How do I stop recommendations becoming repetitive?
Add explicit diversity constraints and recency decay in the re-ranking layer. Relevance alone converges on narrow sets that feel stale quickly.
Want recommendations that move a commercial metric rather than just existing? A free 30-minute consultation will scope the baseline, the data and the test.
