Ranking vs retrieval

Any system that returns a ranked list from a large collection, a search engine, a feed, a product grid, a set of recommendations, answers the same question twice, at two different scales. First: of everything in the corpus, which few hundred items are even worth looking at for this request? Then: given those few hundred, what is the best order to show them in? The first question is retrieval. The second is ranking. They are almost always implemented as two separate stages, and the reason they are separate is the whole point of this piece.

The confusion is understandable, because both stages produce an ordered list and both are, loosely, about relevance. But they optimize for opposite ends of a tradeoff. Retrieval is a recall problem: its job is to make sure the good items are somewhere in the shortlist, and it is judged almost entirely on what it misses. Ranking is a precision problem: its job is to put the best items at the very top, where users actually look, and it is judged on the order near the head of the list. A system can have excellent retrieval and terrible ranking, or the reverse, and the two failures look nothing alike.

The clearest way to hold the difference is by the size of the set each stage operates on. Retrieval runs over the entire corpus, which can be millions or billions of items, so it has a fraction of a millisecond per item and must be cheap. Ranking runs over the shortlist retrieval handed it, a few hundred items, so it can afford a model that costs a thousand times more per item and still fits the request budget. Cheap over everything, expensive over a little. That asymmetry is why you cannot collapse the two into one step.

Retrieval, also called candidate generation, is the stage that reads the entire collection and returns a shortlist. Its defining constraint is that it cannot afford to look at each item carefully, because there are too many of them. So it does not score items one by one with a rich model. Instead it uses index structures that let it jump straight to a small set of plausible matches without touching everything else.

01 inverted indexes

The classic retrieval structure is the inverted index: a map from each term to the list of documents that contain it. To retrieve for a query you look up the query terms and intersect or union their posting lists, which gives you every document that could match without scanning the ones that cannot. Lexical scoring functions such as BM25 add a cheap relevance weight on top, but the index is what makes the step fast. This is still how most keyword search and most follow-graph or tag-based feed retrieval works.

02 approximate nearest neighbor

When items and queries are represented as embedding vectors, retrieval becomes: find the vectors closest to the query vector. Doing that exactly means comparing against every item, which is the cost retrieval exists to avoid. So retrieval uses approximate nearest neighbor (ANN) indexes, structures such as HNSW or IVF that trade a small, bounded amount of recall for a very large speedup. They return most of the true nearest neighbors most of the time, and for a shortlist that is exactly the right trade.

why recall is the metric that matters here

The critical property of retrieval is that it is the only stage that can lose an item permanently. If a genuinely great result never makes it into the shortlist, nothing downstream can put it back: ranking can only reorder what it was given. This is why retrieval is tuned for recall, not precision. It is fine, even expected, for the shortlist to be full of mediocre items that ranking will later push down. It is not fine for the shortlist to be missing the one item the user wanted. A retrieval stage that returns 500 candidates of which only 30 are any good has done its job perfectly, as long as the best 30 are in there.

Ranking takes the shortlist and decides the order. Because it operates on a few hundred items rather than millions, it can spend real computation on each one: a learned model that reads dozens or hundreds of features per item and predicts a score, then sorts by that score. This is where the system earns its precision, and it is precisely the kind of model that would be far too expensive to run over the full corpus.

it predicts a target, not a keyword match

Where retrieval asks "could this item be relevant at all?", ranking asks "how valuable is this specific item to this specific user right now?" and answers with a number. The target is usually a probability such as p(click), an expected engagement value, or a blend like p(click) + w * p(convert). The model learns that target from logged outcomes, which is what lets it capture things a keyword or a cosine distance never could: that this user ignores this author, that this query intent favors recent items, that two weak signals together mean more than either alone.

the model families, in rough order of cost

Ranking models range from a hand-weighted linear score, through gradient-boosted decision trees, to deep networks and, at the top of the funnel of cost, cross-attention rerankers that score the query and item jointly. Trees remain the workhorse for tabular signals because they are cheap and hard to beat. The point is not which family you pick: it is that all of them are affordable only because retrieval already cut the problem down to a shortlist. Run any of them over the whole corpus on every request and the latency budget is gone.

why precision at the top is the metric that matters here

Ranking is judged almost entirely on the head of the list, because that is where attention concentrates. Getting the item at position 200 slightly out of order costs nothing: no one scrolls that far. Getting the wrong item at position one is expensive. So ranking optimizes for precision near the top, and its metrics, unlike retrieval's, are position-weighted. A ranker that surfaces a good-enough answer first beats one that has better items on average but buries the best one on the second screen.

Laid side by side, the two stages differ on nearly every axis that matters. Reading down this list is the fastest way to internalize why they are built, staffed, and tuned separately.

In code the pipeline is short, and the shape of it makes the contract between the stages explicit: retrieval produces candidates cheaply, ranking scores them expensively, and the second stage can only ever reorder what the first one found.

# stage 1: retrieval, cheap, over the whole corpus
def retrieve(query, k=500):
    # index lookups, not a scan: recall over precision
    lexical = inverted_index.match(query)      # BM25 posting lists
    vector  = ann_index.search(embed(query), k)  # approximate NN
    return dedupe(lexical + vector)[:k]     # the shortlist

# stage 2: ranking, expensive, over the shortlist only
def rank(user, candidates):
    feats  = feature_store.batch_get(user, candidates)
    scores = model.predict(feats)              # p(engage) per item
    return sort_desc(candidates, by=scores)

# the contract: rank() can only reorder what retrieve() found.
# a great item missing from `candidates` is gone for good.
def search(user, query):
    return rank(user, retrieve(query))

Because the two stages have opposite objectives, they need different metrics, and picking the wrong one hides exactly the failure you were trying to catch. The rule of thumb is simple: retrieval metrics ignore order, ranking metrics are all about order.

retrieval: recall@k

Retrieval is graded on whether the relevant items made it into the shortlist at all, so its metric is set-based and order-blind. Recall@k is the fraction of all relevant items that appear anywhere in the top k retrieved: recall@k = |relevant ∩ retrieved_k| / |relevant|. It does not care whether a good item is first or fiftieth in the shortlist, only that it is present, because ranking will sort the order out later. You watch recall@k as a function of k to choose the shortlist size: the smallest k at which recall plateaus is the k that feeds ranking the most items worth ranking with the least waste.

ranking: nDCG

Ranking is graded on order, so its metric rewards putting relevance high. nDCG (normalized discounted cumulative gain) discounts each item's relevance by its position, then normalizes against the best possible ordering. Discounted cumulative gain sums the graded relevance of each item divided by a log of its rank, DCG = Σ rel_i / log2(i + 1), so an item at position one counts far more than the same item at position ten. Dividing by the ideal DCG gives a 0-to-1 score you can compare across queries. Mean reciprocal rank and mean average precision are order-aware companions; all of them, unlike recall@k, punish a ranker for burying its best answer.

the two together

Read as a pair, the metrics localize a bad result to a stage. If recall@k is high but nDCG is low, retrieval is finding the good items and ranking is ordering them badly: fix the model. If nDCG on the retrieved set looks fine but users still cannot find things, recall@k is likely the culprit, the item never reached the ranker to be ordered at all. One number cannot tell you which stage to blame; two, one per stage, can.

Almost every recurring failure in this area comes from treating retrieval and ranking as one thing, or from grading one with the other's yardstick. Four show up again and again.

The thread through all four is the same: retrieval and ranking are not two settings of one dial, they are two stages with opposite jobs. Keep them separate in the architecture, in the metrics, and in how you reason about a regression, and most of these mistakes never get made.

Understanding the split is one thing; owning both stages in production is another. Retrieval and ranking have different infrastructure, retrieval is an indexing and serving problem, ranking is a modeling and evaluation problem, and each is a standing commitment rather than a one-time build. Indexes need rebuilding as the corpus turns over; models drift and need retraining from fresh logs. If relevance is your core product surface, a search company, a large marketplace, a social feed, you should build both, because the order of results is exactly where you compete and outsourcing a competitive edge rarely pays off.

The honest case for buying is narrower and specific to which stage is your differentiator. Retrieval is often the part teams genuinely need to own, because it encodes their corpus and their notion of what can match. Ranking is often the part that is necessary but not differentiating, and it is the part with the heaviest ongoing modeling and evaluation cost. If you already retrieve a solid shortlist and just need it ordered well, a decisioning API lets you send those candidates and get a ranked order back, with sponsored slots resolved in the same call. That is the seam Gortex sits in: the feed ranking API takes the candidates you already retrieve and returns the ranked order, and solutions shows the same decisioning surface across adjacent ranking problems. Build the retrieval that is yours, buy the ranking layer when it is necessary but not the thing you are selling.