Recommendation engine architecture
A recommender picks a handful of items out of a catalog of millions and puts them in front of one person. That is a retrieval problem and a ranking problem stacked on top of each other. This is how the two stages fit together: where candidates come from, how they are scored, what the serving stack has to guarantee, and how you know any of it is working.
Retrieve a shortlist, rank it, re-rank for the surface.
A recommendation engine answers a discovery question: of everything in the catalog, which items should this user see, and in what order. The catalog is the whole space, products, videos, tracks, listings, so the engine cannot score it directly. It runs the same three-stage shape a feed uses, but the weight sits in a different place.
A feed usually starts with a naturally scoped set: recent posts from accounts you follow, plus some injected content. Recency and the follow graph do half the work before ranking begins. A recommender rarely has that scoping. There is no follow graph telling you which of ten million products matter to this user right now, so candidate generation carries more of the load, and getting a good item into the shortlist is where most of the outcome is decided.
01 candidate generation
Also called retrieval. Several cheap sources each nominate a few hundred items, and their union becomes the shortlist, typically a few hundred to a couple thousand out of millions. Each source has a different bias: collaborative signals surface what similar users liked, embedding retrieval surfaces what is semantically close to a user's taste, content sources surface items by their attributes. The goal is recall. An item missed here is gone, because nothing downstream reconsiders it.
02 ranking
The expensive, precise step. A learned model scores every candidate against this user in this context and sorts by the score. Because it runs over hundreds of items rather than millions, it can afford rich features and a heavier model. For a recommender the score is usually multi-objective: not just will they click, but will they buy, finish, subscribe, come back.
03 re-ranking and serving
The final pass turns a sorted list into what ships. It enforces the rules a raw relevance score ignores: diversity so the row is not eight variants of one product, business constraints like inventory or region, deduplication against what the user already saw, and sponsored placements. Then it returns the order and logs every decision for the next model to learn from.
Several weak retrievers beat one clever one.
No single retrieval source is complete. Collaborative filtering is blind to brand-new items. Embedding retrieval can drift toward the generic center of a user's taste. Content matching ignores behavior entirely. The standard design runs them in parallel and unions the results, so a weakness in one is covered by another. Three families do most of the work.
ALS learns latent user and item vectors whose dot product approximates
a rating. Powerful on dense catalogs, useless for an item nobody has touched yet.
dot(user, item) in a few milliseconds. This is the generalizing workhorse of modern
retrieval.
The two-tower design is worth seeing concretely, because its shape explains why retrieval is fast. All the expensive item embedding happens offline; the online path is one user embed plus an index lookup.
# item vectors are embedded offline and indexed once # online we only embed the user, then search the index def retrieve(user, k=800): uvec = user_tower(user) # one forward pass emb = ann_index.search(uvec, top=400) # nearest items, ~5ms # union with cheaper precomputed sources cf = item_to_item(user.recent_items, top=300) cont = by_attributes(user.affinities, top=200) cands = dedupe(emb + cf + cont) # overlap is expected return cap_per_source(cands, k) # no source dominates
Two rules keep the union honest. Deduplicate, because good items surface from more than one source and you do not want them counted three times. Cap each source's contribution, so a single prolific retriever cannot crowd the others out and quietly narrow what the ranker ever gets to consider.
Score every candidate on the outcome that pays.
Retrieval hands the ranker a shortlist with no reliable order: an item can appear because one weak source liked it. The ranker's job is to impose a real order by predicting, for each candidate, how valuable showing it is to this user right now. Unlike a feed that often optimizes a single engagement target, a recommender usually cares about several outcomes at once, and they trade off.
multi-objective scoring
A click is not a purchase, and a play is not a finished episode. The common pattern is a multi-task model with a
head per objective, then a weighted blend into one comparable number. On a marketplace that blend is naturally
expected value: score = p(click) * p(buy | click) * value, so a cheap item everyone
clicks does not beat a considered purchase. On a media surface it leans toward completion and long-term
retention rather than the click that starts a video and abandons it.
model families
In rough order of cost: gradient-boosted trees, then deep networks. Trees are the sensible baseline and are hard
to beat on tabular features. Deep architectures earn their keep when embeddings and high-cardinality sparse
features carry the signal, which is common in recommendation. A wide-and-deep model pairs
a linear wide part that memorizes specific user-item affinities with a deep part that generalizes across sparse features,
while a DLRM-style model drops the wide part entirely and instead learns explicit
pairwise interactions between feature embeddings.
calibration is not optional here
A feed can often get away with scores that only need to sort correctly. A recommender usually cannot, because
the score feeds arithmetic downstream: expected-value ranking multiplies probabilities by price, and sponsored
placement compares an organic score against a bid. If p(buy) reads 0.3 but the real rate
is 0.1, every product of it is wrong. Calibrate the heads, on a held-out slice, so a predicted probability means what
it says.
Two stages exist so the request can stay cheap.
The retrieve-then-rank split is not academic tidiness, it is what makes the request affordable. Scoring millions of items per request with a rich model is impossible inside a latency budget; scoring a few hundred is routine. Retrieval trades precision for a cheap pass over everything, ranking trades breadth for precision over a little. The stack underneath has to serve both without the two disagreeing about what a feature means.
Feature reads dominate the online budget, so cache with the grain of the data. Item features and item embeddings are shared across every user who sees that item and cache well across the fleet; user features change slowly within a session and cache well per user. Read the whole candidate set's features in one batched call, never one call per item, and the ranking pass itself stays the small part of the budget.
Measure the stages separately, then decide online.
A recommender has two stages that fail in different ways, so evaluate them separately before you evaluate the whole. A bad number at the end is ambiguous: it could be retrieval never surfacing the right item, or the ranker ordering a good shortlist badly. Splitting the metric tells you which.
offline: retrieval then ranking
Retrieval is judged by recall at k: of the items a user actually engaged with, how many were in the candidate
set the retriever produced. If the eventual positive is not retrieved, no ranker can save it, so this is the
ceiling on everything downstream. Ranking is judged by order-aware metrics on the retrieved set:
nDCG (normalized discounted cumulative gain), which rewards putting good items near the
top where users look, with mean average precision alongside. Both are proxies, which is why they only filter candidate
models rather than crown one.
online: A/B tests and the metrics a recommender needs
Online you split traffic and compare on the outcome the surface exists for: purchases, subscriptions, time well spent, retention. A recommender needs two guardrails a feed can under-weight. Diversity, so the engine does not collapse onto one narrow slice of taste and stop being useful. Catalog coverage, the share of the catalog that ever gets recommended, because an engine that only ever surfaces the popular head starves the long tail that made the catalog worth having. A model that lifts clicks while coverage craters has not won, it has just gotten narrower.
The stack has to work before it has data.
Every behavioral signal a recommender leans on is empty for something new, and cold start is where the collaborative parts of the system are useless by construction. It shows up in two forms, and a recommender feels the second one far more than a feed does.
01 new user
No history, so collaborative and two-tower retrieval have nothing personal to work with. Fall back to what does not need history: content and context features, popularity priors, and any onboarding signal you can collect cheaply, a few taste picks or the entry point they arrived through. The goal is to gather enough interaction, fast, to hand the personalized path something to stand on.
02 new item
This is the structural one. A feed's cold start is mostly new users; a recommender ingests new items continuously as the catalog churns, and a new product or track has zero interactions the moment it needs to be discoverable. Content embeddings make it retrievable from day one, but retrievable is not the same as ranked, so give new items a measured exploration budget: show them enough, in enough contexts, to earn the interaction history the ranker needs. A contextual bandit formalizes this, spending a slice of traffic to learn an item's value instead of assuming the current model already knows it. Without a deliberate exploration budget the engine recommends what is already popular and never learns anything new is good, the feedback loop that quietly kills a catalog.
When to own the stack, and when not to.
Everything above is buildable, and if recommendation is the product you should probably build it. When discovery is how users find everything they consume or buy, the quality of the retrieval and ranking stack is a large part of what you are actually selling, and outsourcing that is rarely wise. If you have the ml and infrastructure headcount and the engine is the differentiator, own it end to end.
The honest case for buying is narrower and real. A recommender is not a project that finishes: retrieval indexes need rebuilding, embeddings and models drift, cold start never stops arriving, and every new surface needs its own tuning, so the cost is a standing one rather than a one-time build. If recommendation is necessary but not your differentiator, or you need a working engine before you can justify the team to run one, a decisioning API lets you send the candidates you already retrieve and get a ranked, calibrated order back, with sponsored slots resolved in the same call. That is the seam Gortex sits in. The recommendation engine API covers the ranking and serving layers described here, and solutions shows the same decisioning surface for adjacent problems. Buy to get moving or to skip undifferentiated work, build when the order of the recommendations is the thing you are selling.