Recommender Systems

1A table of people and items

Stay with the cinema-club sheet. Four members — Ananya, Bharat, Chitra, Dev — and five films — Dune, Arrival, Inception, Amelie, Parasite. Read Ananya's row left to right: 5, 5, 4, 1, and a blank. Those numbers are ratings she already gave; the blank is Parasite, a film she has not seen. The job is not to invent a sixth film. It is to fill blanks like that and hand Ananya a ranked list of items for one person, not one rent for a flat.

Each cell is one person-and-film pair. A 5 means she loved it; a 1 means she did not; a blank is missing, not a zero. Bharat's Parasite cell is 1 — he saw it and shrugged. Everything below uses this same sheet. Machine Learning's 200-flat rent notebook is a different table: each row was one flat and the answer was already written. Neural recommenders that eat this club sheet wait for Deep Learning; here the tools are a popularity baseline, item tags, and neighbours whose rows look like yours.

Figure. The cinema-club sheet itself: four members, five films, numbers 1–5, and a blank. Ananya's Parasite cell is the thing to fill. The 200-flat rent notebook is a different table — one number per row, not a ranked list for one person.

2Tags versus other people

There are two classical families, and they read the club sheet from opposite sides. Content-based filtering describes each film with tags you already know — Dune, Arrival and Inception are sci-fi; Amelie is a romance; Parasite is a drama — then asks what Ananya already liked. Her sci-fi ratings are 5, 5 and 4, her romance rating is 1, so a content model proposes more sci-fi and stays away from romance. It never looks at Bharat's row.

Collaborative filtering ignores the tags and reads the other people. It asks: whose other ratings look like Ananya's? Those people are her neighbours. Then it copies what they did on the blank cell. Bharat also loved the three sci-fi films and shrugged at Amelie, so he is a neighbour; he rated Parasite 1, and that 1 becomes Ananya's collaborative guess. The two families can agree or fight; they are different evidence, not two names for the same method.

Figure. The same blank — will Ananya like Parasite? — is answered two ways. Content reads the drama tag against her sci-fi mean of 4.67. Collaborative reads Bharat, whose row looks like hers, and copies his 1.

1. Content-basedTag each item, then match those tags to the person's past ratings on the same tags.
2. CollaborativeFind people whose filled cells look like this person's, then read their rating on the blank item.
3. Do not mix the namesA tag model never needs another member; a neighbour model never needs the poster.

3A neighbour is a similar row

A neighbour is not a friend on the app. It is a person whose rating row points the same way as yours on the films you both saw. To compare Ananya and Bharat you drop Parasite (she has no rating there) and keep the four shared films, in order: Dune, Arrival, Inception, Amelie. Ananya's four numbers are 5, 5, 4, 1. Bharat's four numbers are 5, 4, 5, 2. Those two short lists are the only inputs to the comparison.

Cosine similarity is the cosine of the angle between those two lists: 1 means they point the same way, 0 means they are unrelated. Their cosine is 0.978 — almost the same direction. Ananya versus Chitra on the same four films is only 0.527: Chitra liked what Ananya did not. Once you have those scores, start from the nearest neighbour's rating on the blank. Bharat is nearest and rated Parasite 1, so the neighbourhood guess for Ananya is 1. You can later average several neighbours, weighted by similarity; the first skill is seeing that the guess comes from people, not from the film's poster.

Figure. Similarity is drawn as closeness to Ananya, not as friendship. Bharat at 0.978 is the nearest row; Dev at 0.606 is farther; Chitra at 0.527 is the least like her on the films they both rated. The Parasite guess starts from Bharat.

1. Keep shared filmsDrop any film either person left blank. Cosine on a padded 0 treats unrated as hated.
2. Score the angleCosine of the two short lists: nearer to 1 means the rows point the same way.
3. Copy the nearest ratingThe nearest neighbour's number on the blank item is the first personalised guess.

4The club favourite is a baseline

Before any neighbour search, there is a one-line baseline: ignore who is asking, and score each film by the average of the stars it already has. On the club sheet that is a column mean over the filled cells only — blanks are not zeroes. Parasite's observed ratings are Bharat 1, Chitra 5, Dev 5, so its popularity is (1+5+5)/3 = 3.67. Arrival also sits at 3.67; Inception is lowest at 2.75. A popularity recommender hands every member the same ranking.

That number is useful and easy to beat. Useful, because a film nobody liked is rarely the right first suggestion. Easy to beat, because popularity ignores who is asking. Ananya's nearest neighbour rated Parasite 1, so a personalised guess is 1 while the baseline says 3.67. If you ship the baseline and call it a recommender, Ananya gets the club favourite instead of a film for her.

Figure. Each bar is that film's mean over filled cells only: Dune 3.25, Arrival 3.67, Inception 2.75, Amelie 3.00, Parasite 3.67. Parasite ties Arrival as a club favourite, which is why a baseline would offer it to Ananya — her nearest neighbour's 1 is the personalised contradiction.

1. Average the filled starsColumn mean over observed ratings only. A blank does not pull the mean toward zero.
2. Same list for everyoneRank films by that mean. Ananya, Chitra and a stranger receive the identical order.
3. Keep it as a baselineCompare every personalised guess to this number. Beating it is the first proof the neighbour was worth computing.

5Empty rows and empty columns

Collaborative filtering and the popularity baseline both starve when a row or a column is empty. A new member, Esha, has no ratings yet: there is no vector to compare to Ananya or Bharat, so she has no neighbours, and a column mean still does not know what she likes. A new film, say a sci-fi title with no ratings yet, is just as blank: nobody's row contains a number for it, so user–user collaborative filtering cannot score it, and popularity has no stars to average.

Content-based filtering can still move. If the new film is tagged sci-fi, and Ananya's sci-fi mean on Dune, Arrival and Inception is (5+5+4)/3 = 4.67, a content model can propose it to her on day one. The swap is the other cold start: a brand-new person with no history gives content nothing to match either, unless you ask for a few tags up front. The practical rule: collaborative filtering needs overlapping ratings; content-based filtering needs a description of the item and some taste on those descriptions.

Figure. Ananya already has three sci-fi ratings, so a content model can propose a brand-new sci-fi film. Collaborative filtering cannot: that film has no stars, so no neighbour has a number to copy. Esha sits alone with an empty row — neither family has a history to read unless she names a few tags.

6Lab: popularity versus a neighbour

The notebook rebuilds the club sheet as a 4×5 array. A 0 is unrated, not a zero-star review — that is why every cosine is computed only on the films both people actually rated, and why popularity is a column mean over the positive cells only.

Cell 1 prints each film's popularity, Ananya's cosine with Bharat and with Chitra, and Bharat's Parasite rating (the neighbourhood guess). You should see popularity 3.67 for Parasite next to neighbour_guess_parasite 1 — the same disagreement the lesson just computed by hand.

Cell 2 draws those popularity means as a bar chart so the two 3.67 films (Arrival, Parasite) stand next to Inception at 2.75. The picture is the baseline, not Ananya's list.

No diagram — the popularity bars are drawn by the coding lab plot, not a static figure.

Coding lab. Club-sheet popularity and neighbour cosine runs in the app, with checks on your output.