# Taste-Profile Recommender -- Full Technical Write-Up

## Citation

Koren, Y., Bell, R., and Volinsky, C. 2009. Matrix Factorization Techniques for Recommender Systems. IEEE Computer 42, 8 (2009), 30-37. DOI: 10.1109/MC.2009.263.

## What this document is

This is the fully rigorous, non-interactive companion to the demo's guided
walkthrough served at `/` -- the SAME material, written for a technical reader
who wants the complete formal treatment rather than the plain-English,
drag-it-yourself version. (Documentation Depth requirement, g-dev-poc
SKILL.md Step 4: a plain-English walkthrough and a separate, one-click-away
rigorous layer, not one paragraph asked to serve both readers.)

## Methodology (plain-English summary)

Matrix factorization predicts how a user would rate an item by breaking that rating into four learned pieces, then adding them together: (1) the GLOBAL AVERAGE rating across everyone, (2) a USER BIAS -- some users rate everything higher or lower than average, (3) an ITEM BIAS -- some items get rated higher or lower than average regardless of who's watching, and (4) a PERSONALIZED MATCH -- each user and each item is assigned a vector of 20 numbers ('latent factors'), learned purely from the ratings data with no hand-labeled genres or categories, and the match score is how well the user's vector and the item's vector point in the same direction. A user whose vector points toward 'the scifi/action corner' of that 20-dimensional space gets high match scores on items whose vectors point the same way -- even though the model was never told what 'scifi' or 'action' means. Below, five independently-manipulable controls put both that four-term equation AND the falsification proof itself in your hands: 'Drag the taste vector' places the gold dot at a real user's real learned position on 2 of the model's 20 latent dimensions and recomputes the personalized-match term live as you drag it; the AXIS PICKER lets you choose *which* 2 of the 20 dimensions get plotted, so you aren't stuck with one fixed projection -- every one of the 20 is a real, explorable axis; the USER BIAS slider and ITEM BIAS slider let you override those two additive terms directly and watch the predicted rating -- and the ranked list -- recompute live; and 'Verify the proof yourself' lets you draw random pairs from the ACTUAL held-out set the proof below was scored on and watch your own running RMSE converge onto that pre-registered number as you draw more -- the same Law-of-Large-Numbers pattern that makes a coin's observed frequency converge to its true probability. All five read from the SAME formula and the SAME learned numbers the server uses for real recommendations and the real proof, not a canned example.

## The prediction equation

    r_hat(u, i) = mu + b_u + b_i + q_i . p_u

    mu            global mean rating over the training set (2.2188 for this run)
    b_u           user u's learned bias term
    b_i           item i's learned bias term
    q_i, p_u      item i's / user u's learned latent-factor vectors, each in R^20
    q_i . p_u     dot product of those two vectors (the "personalized match" term)

## The objective function

Every b_u, b_i, q_i, p_u above is fit jointly by minimizing one regularized
squared-error loss over the known ratings K:

    L = sum over (u,i) in K of (r_ui - r_hat(u,i))^2
        + lambda * (b_u^2 + b_i^2 + ||q_i||^2 + ||p_u||^2)

    lambda = 0.05  (this run's regularization weight -- keeps every learned
                    number small and discourages overfitting to any one rating)

## The update rule (stochastic gradient descent)

For every known rating, visited in random order, each pass:

    e    = r_ui - r_hat(u, i)
    b_u += lr * (e - lambda * b_u)
    b_i += lr * (e - lambda * b_i)
    q_i += lr * (e * p_u - lambda * q_i)
    p_u += lr * (e * q_i - lambda * p_u)

    lr = 0.01  (learning rate, "gamma"), repeated for 60 full passes
    ("epochs") over the training data. p_u on the right-hand side of the q_i
    update is the value from BEFORE this interaction's own update (Koren et al.
    2009's simultaneous-update formulation) -- see recommender/mf.py for the
    exact, unmodified implementation this document describes.

## The proof metric (RMSE)

    RMSE = sqrt( (1/|T|) * sum over (u,i,r) in T of (r_ui - r_hat(u,i))^2 )

Computed on T, the held-out set: 2105 (user, item, rating)
triples never used during training.

    held-out RMSE   = 0.8719
    threshold       = 1.1867  (popularity-only baseline's own RMSE)
    outcome         = PASS

## The self-deception risk this proof guards against

Popularity bias: a recommender that has simply learned "surface whatever is
already popular" can look accurate on many real-world datasets, because
popular items collect more (and less noisy) ratings. See Abdollahpouri et al.,
"The Unfairness of Popularity Bias in Recommendation"
(https://arxiv.org/abs/1907.13286) for a survey of the effect. This demo's
falsification gate compares against a popularity-only baseline computed on the
SAME held-out set (recommender/train.py's _popularity_baseline) -- beating it
is direct evidence the model learned real per-user taste signal, not just which
items are broadly liked.

---
Generated from this run's own artifacts (proof_result.json, output/model.json)
-- not retyped by hand.