How to build a feed ranking system

A feed ranking system answers one question on every request: of all the items this user could see right now, in what order should they see them? That question is too large to answer in a single step, so it is split into three stages that run in sequence, each one narrowing the problem for the next.

01 candidate generation

You cannot score every item in the catalog on every request. A platform with tens of millions of posts scores a few hundred to a few thousand. Candidate generation, also called retrieval, is the fast, cheap step that pulls that shortlist: recent posts from people you follow, items similar to ones you engaged with, a slice of trending content, and some exploration. The goal here is recall, not precision. Miss a good item at this stage and no downstream model can recover it.

02 ranking

Ranking is the expensive, precise step. It takes the candidate shortlist and assigns each item a score that predicts how valuable it is to this user in this context, then sorts by that score. This is where a learned model earns its cost, because it runs over hundreds of items rather than millions.

03 serving

Serving turns the sorted list into a feed. It applies business rules that a pure relevance score does not capture: diversity so the top of the feed is not five posts from one author, freshness floors, blocked content, and sponsored slots. Then it returns the final order and logs what it did.

The three stages map onto a small set of long-lived components. Naming them separately matters, because each one has a different scaling profile, a different failure mode, and usually a different team.

The dependency the logging component creates is worth stating plainly. The model is trained on decisions the previous model made. The quality of tomorrow's ranking is bounded by how honestly you log today's.

Do not begin with a neural network. The first version of almost every good feed is a weighted sum of a handful of signals: recency, a follow-graph affinity, and a popularity term. It is legible, it is fast, and it gives you a baseline that a learned model has to beat before it is worth its operational cost. Many surfaces never need more than a well-tuned heuristic.

from heuristics to a learned ranker

The move to a learned ranker happens when the number of signals grows past what you can weight by hand and their interactions start to matter. A learned model predicts a target, typically a probability such as p(engage) or a blend like p(click) + w * p(dwell), from features. The families you will meet, in rough order of cost: logistic regression, gradient-boosted trees, and deep networks. Trees are the workhorse: they handle mixed feature types, they are cheap to serve, and they are hard to beat on tabular signals.

features

Features come in three shapes. User features describe who is asking: their history, their languages, their session context. Item features describe what is being ranked: age, author, topic, historical engagement rate. The most valuable features are the crossed ones, user-by-item: has this user engaged with this author before, does this topic match their recent sessions. Crossed features are also the most expensive to compute on the request path, which is the tension the serving layer manages.

training data is your own logged decisions

You do not label feed ranking data by hand. The labels come from what users did with what you showed them: a shown item that was clicked is a positive, a shown item that was skipped is a negative. This is why logging is load-bearing, and it is also where the subtle bugs live. You only observe outcomes for items you chose to show, so the model learns from a feed the previous model shaped. Section 07 returns to this.

Ranking runs while a user waits for their feed to load. That makes latency a hard constraint, not a metric to improve later. Treat the end-to-end budget, often somewhere near 200ms at p99, as a fixed sum you allocate across retrieval, feature fetch, and scoring. Every new signal spends from it.

The scoring loop itself is the simple part. What surrounds it, batching, caching, and a degradation path, is what keeps it inside budget under real load.

# score every candidate, then order the feed
def rank(user, candidates):
    # one batched read, not one call per item
    feats  = feature_store.batch_get(user, candidates)
    scores = model.predict(feats)          # p(engage) per item
    ranked = sort_desc(candidates, by=scores)

    # business rules: diversity, freshness, sponsored slots
    ranked = apply_rules(ranked)
    return ranked

# if a dependency blows the budget, degrade, do not fail
def serve(user, req):
    try:
        cands = retrieve(user, req, timeout_ms=40)
        return rank(user, cands)
    except Timeout:
        return cached_or_recency_feed(user)  # stale beats blank

caching and fan-out

Feature reads dominate the budget, so cache them. User features change slowly and cache well across a session; item features are shared across every user who sees that item and cache well across the fleet. Fan-out, calling retrieval sources in parallel rather than in series, keeps the slowest source from setting the total. Read features for the whole candidate set in one batched call, never one call per item.

degradation

Something will be slow at peak. The serving layer needs an answer for that which is not a blank feed. A stale cached order, or a plain recency sort when the model times out, is almost always better than an error. Decide the fallback deliberately and rehearse it, because it will run in production whether you designed it or not.

Two kinds of evaluation do two different jobs. Offline metrics, computed on logged data, are a cheap filter: they tell you which model versions are worth putting in front of users. Online experiments are the decision: they tell you whether a change actually helped. Neither replaces the other.

offline: ranking metrics

Offline you measure how well the predicted order matches observed engagement. Ranking-aware metrics reward putting good items near the top, where users actually look. nDCG (normalized discounted cumulative gain) is the common one: it discounts relevance by position, so an item ranked first counts for more than the same item ranked tenth. Mean average precision and recall at k are useful companions. All of them are proxies, which is why they only filter.

online: A/B tests and guardrails

Online you split traffic and compare. The primary metric is whatever the feed is for: sessions, time well spent, retention. Alongside it you watch guardrail metrics, quantities that must not get worse even if the primary metric improves: latency, block and report rates, creator-side reach, revenue. A model that lifts engagement while raising the report rate has not won, it has moved the cost somewhere you were not looking.

A feed that ranks well is valuable inventory, and at some point a sponsored item wants a place in it. The wrong way is to bolt on a second system that jams paid content into fixed positions regardless of fit. That trains users to distrust the feed, and distrust is expensive to win back.

The better model treats a sponsored candidate as another item competing for a slot, scored on a common currency. Put both on the same expected-value-per-impression scale: an organic item by its predicted engagement value, v_organic = E[value | show], and a sponsored item by v_ad = bid * p(click), the advertiser's expected payment. Because both are value per impression, they compare directly: a paid item wins a slot only when v_ad clears the v_organic it would displace, so relevance sets a floor that money cannot walk under. Cap sponsored density per screen, resolve placement in the same pass that produces the organic order, and you get monetization that does not fight the ranking it sits inside.

Everything above is buildable, and if ranking is your core product surface you should probably build it. The order of a feed is where a social network or a large marketplace competes, and outsourcing your competitive edge is rarely wise. If you have the ml and infrastructure headcount and the feed is the product, build it.

The honest case for buying is narrower and real. The system never finishes: models drift, signals shift, and every new surface needs its own tuning, so the cost is a standing one, not a one-time build. If ranking is necessary but not your differentiator, or you need a working feed before you can justify a ranking team, a decisioning API lets you send the candidates you already retrieve and get an order back, with sponsored slots resolved in the same call. That is the seam Gortex sits in. The feed ranking API covers the ranking and serving layers described here, and solutions shows the same decisioning surface for adjacent problems. Buy to get moving or to skip undifferentiated work, build when the order of the feed is the thing you are selling.