Review Sentiment Application

1Short text labels

Every app store and shop page collects reviews faster than any team can read them. Sentiment analysis is the job of deciding, from the text alone, whether the writer sounds happy or unhappy — so that support and product teams can triage: route each incoming review into a positive queue or a negative queue, and read the angry ones first. That is the whole decision this lesson builds a model for: one short piece of text arrives, and it lands in one of two buckets. In machine learning terms this is binary classification — 'binary' because there are exactly two possible answers, 'classification' because the answer is a category rather than a number.

Before any model, the label needs a policy. The label is the answer column: for each review in your training file, a 1 if it is positive and a 0 if it is negative. If your reviews come with star ratings, a common policy is stars of 4 or 5 mean positive and 1 or 2 mean negative — but notice that leaves 3-star reviews undecided, and 'stars ≤ 2' is genuinely not the same question as 'mentions a defect' (a 5-star review can still report a broken hinge). Whoever builds the file must write the policy down and apply it consistently, because the model can only learn the question the labels actually answer. One more rule: the model may only read text that exists at prediction time. A support agent's resolution notes are written after the ticket closes, so gluing them onto the review text would train a model on information it will never have when a fresh review arrives.

To make every step of this lesson checkable by hand, we will carry three tiny reviews through the whole topic and do all the arithmetic on them. R1: "battery dies fast" — labelled negative. R2: "great battery life" — labelled positive. R3: "fast shipping" — labelled positive. Three reviews, three words or fewer each. They are small enough that when we count words, weight them, and score them, you can verify every number yourself — and that is exactly what the next concepts do.

The framing in words: a short review arrives, the model reads only its text, and the review lands in the positive or the negative queue — so humans read the angry ones first.

2Bag-of-words: text becomes counts

A model like the ones in this course fits numbers — it cannot fit a line through the word "battery". So the first real step is featurisation: turning each review into a row of numbers. The oldest and still remarkably strong recipe is called bag-of-words, and the name is literal: imagine tipping all the words of a review into a bag, so their order is lost, and then just counting how many times each word landed in the bag. Before counting, the text is tokenised — split into words, lowercased, punctuation stripped — so that "Great!" and "great" become the same token. A token is simply a cleaned-up word.

Counting needs a fixed list of slots, and that list is the vocabulary: every distinct token that appears anywhere in your training reviews, in a fixed (say alphabetical) order. Across our three reviews the vocabulary has exactly six words: battery, dies, fast, great, life, shipping. Each review then becomes a count vector — one slot per vocabulary word, holding how many times that word occurs in that review. Read R2 "great battery life" against the six slots: battery appears once (1), dies not at all (0), fast not at all (0), great once (1), life once (1), shipping not at all (0). So R2 becomes the vector (1, 0, 0, 1, 1, 0). Every review in the file gets its own such row, and the table below shows all three.

Notice how many zeros there are: even in this toy corpus, half of R2's slots are empty. On a real corpus the vocabulary has hundreds or thousands of words, while any single review uses only a handful — so almost every slot in every row is zero. A vector that is mostly zeros is called sparse, and libraries store only the non-zero entries, which is why a 10,000-word vocabulary does not blow up memory. The worked example below does this arithmetic.

The figure is the count table below: one row per vocabulary word, one column per review, each cell the number of times that word occurs in that review — R1 and R2 share the battery row, and only R1 has a count in the dies row.

3TF-IDF

Raw counts treat every word as equally informative, and they are not. In R1 "battery dies fast", the words battery and dies both count 1 — but battery also appears in the glowing R2, so it signals the topic (this review is about the battery), not the mood. Dies appears in the negative review and nowhere else: that is the sentiment carrier. We want each word's number to reflect not just how often it occurs, but how distinctive it is.

TF-IDF builds that number from two ingredients. Term frequency, tf, is how often the word occurs in this one review — in our three-word reviews every tf is 1. Document frequency, df, is how many reviews in the corpus contain the word at all: df(battery) = 2 (it is in R1 and R2), df(dies) = 1. Inverse document frequency turns df into a rarity score, \mathrm{idf}(t) = \ln\frac{N}{\mathrm{df}(t)}, where N is the number of reviews (here 3) and \ln is the natural logarithm. Read its behaviour at the extremes: a word in every review gets \ln(3/3) = \ln 1 = 0 — a word present everywhere separates nothing, so it is worth zero — while a word in just one review gets the largest value, \ln(3/1) \approx 1.10.

The full weight is the product, \text{tf-idf}(t, d) = \text{tf}(t, d) \times \mathrm{idf}(t): high when a word is frequent in this review and rare across the corpus. The worked example computes it for battery and dies; fast, which like battery appears in two of the three reviews, gets the same weight as battery, 1 \times \ln(3/2) \approx 0.41. One practical note: sklearn's TfidfVectorizer uses a slightly smoothed idf formula and then rescales each row to unit length, so its numbers differ a little from this hand version — but the story is identical: distinctive words up, everywhere-words down.

Figure. TF-IDF weights for the three words of R1 "battery dies fast", computed by hand in the worked example: battery and fast each appear in two of the three reviews, so idf = ln(3/2) gives them 0.41; dies appears only in the negative review, so idf = ln(3/1) lifts it to 1.10. The sentiment carrier now towers over the topic words.

4A linear classifier on word weights

Now the review is a sparse vector of TF-IDF weights, and a linear classifier — LogisticRegression in sklearn — turns it into a verdict. What it learns during training is disarmingly simple: one number per vocabulary word, called that word's weight (or coefficient). A positive weight means the word's presence is evidence the review is positive; a negative weight means evidence it is negative. To score a review, the model multiplies each present word's TF-IDF value by that word's learned weight and adds everything up. Words absent from the review contribute exactly zero — which is why sparsity is a gift: only the handful of non-zero slots do any work.

The sum, usually written z, can come out as any number — 3.2, or −1.86. To turn it into something a triage queue can use, the logistic function squashes it into a probability between 0 and 1: p = \frac{1}{1 + e^{-z}}, read as the model's probability that the review is positive. A large positive z gives p near 1, a large negative z gives p near 0, and z = 0 gives exactly 0.5 — the fence. The standard rule then routes: p \ge 0.5 to the positive queue, below it to the negative queue.

Suppose training on a big pile of labelled reviews landed on these (illustrative, but realistically shaped) weights for R1's three words: dies −1.8, battery −0.1, fast +0.4 — a strongly negative word, a near-neutral topic word, a mildly positive word. The worked example scores R1 with them, and the payoff generalises: after fitting any text classifier, print the most negative and most positive weights and read them. If the top negative words look angry and the top positive ones look delighted, the model has learned something a human recognises; if garbage tokens carry big weights, you have found a data problem before it found you.

Figure. R1 'battery dies fast' scores by weight × TF-IDF: battery −0.1 × 0.41 = −0.04, dies −1.8 × 1.10 = −1.98, fast +0.4 × 0.41 = +0.16. Sum z = −1.86. Then p = 1/(1+e^{1.86}) ≈ 0.13, which is below 0.5, so the review routes to the negative queue. One strong word outvoted two milder ones. Boxes are equal chips, not to scale.

1. One weight per wordTraining assigns every vocabulary word a coefficient: positive words push toward the positive queue, negative words the other way.
2. Multiply and addFor the words the review contains, sum weight × TF-IDF. Absent words contribute zero — sparsity keeps this cheap.
3. Squash and routeThe logistic function turns the sum into p(positive); at or above 0.5 the review routes positive, below it negative.

5Fit the vectoriser on train only

Here is the step where careful people quietly cheat. A TfidfVectorizer is fitted, exactly like a model: from the data it is shown, it learns the vocabulary (which words get slots) and every idf value (how rare each word is). So it matters enormously which data it is shown. If you vectorise the whole file first and split into train and test afterwards, the test reviews have already shaped the features — their words are in the vocabulary, their document counts are inside every idf. The held-out score then no longer measures performance on unseen data, because the test rows helped build the machine that reads them. That is leakage, and it always errs in the flattering direction.

The fix is to chain the vectoriser and the classifier into a single Pipeline object — in sklearn, make_pipeline(TfidfVectorizer(...), LogisticRegression(...)). Now one call to fit(X_train, y_train) fits both stages on training rows only, and one call to score(X_test, y_test) transforms the test reviews using the train-built vocabulary and idf before scoring them. A test-set word that never occurred in training simply gets no slot — which sounds like a flaw and is actually the honest behaviour, because tomorrow's real reviews will also contain words today's model has no weight for. One more knob belongs here: max_features=200 tells the vectoriser to keep only the 200 most frequent words, which caps the matrix size — in a browser teaching runtime that cap is what keeps the lab snappy, and on real corpora it drops one-off typos that would otherwise each claim a column.

Figure. The leakage boundary: after the split, everything the pipeline learns — vocabulary, idf values, classifier weights — comes from the 90 training rows alone. The 30 test rows cross to the right exactly once, transformed by the train-built features and scored. Vectorise before splitting and the test rows would sit inside the left box too.

1. Split firstCut train and test before anything is fitted — the vectoriser counts as something that is fitted.
2. Fit inside the pipelinemake_pipeline(vectoriser, classifier); fit(X_train, y_train) builds vocabulary, idf and weights from training rows only.
3. Score the held-out rowsscore(X_test, y_test) transforms test text with the train-built features and reports accuracy on rows nothing was fitted on.

6Lab: sentiment baseline

Time to run the whole lesson as sixteen lines of code. The dataset is /data/review_sentiment_v1.csv, a synthetic teaching file of 120 rows and two columns: text, a short review phrase, and label, a 1 for positive and a 0 for negative. It is exactly balanced — 60 positive rows and 60 negative — and the phrases are short, around five or six words each, in the same spirit as our three running reviews: "terrible quality broke quickly" sits next to "loved the quality highly recommend". Because the file is generated for teaching, it is deliberately clean: every positive row carries clearly positive words and every negative row clearly negative ones, with no sarcasm, no typos, and no mixed feelings. Keep that in mind for when the score prints — it explains what you will see.

Walk the code before running it. pd.read_csv loads the file into a table called df, and train_test_split hands the text column and the label column in together, so every review stays paired with its own label through the shuffle. The four names it returns follow the convention used all over scikit-learn: the capital X marks inputs (here, review text), the small y marks answers (the 0/1 labels), and the suffix says which pile the piece belongs to — X_train and y_train are the rows the model will study, X_test and y_test the rows locked away for the honest measurement. test_size=0.25 holds out a quarter of the rows, so the piles come out at 90 training reviews and 30 test reviews — the first print line shows exactly those two numbers, and checking them is worth the two seconds, because a mistyped fraction here quietly changes every number after it. random_state=0 pins the shuffle so every run (yours today, yours tomorrow, a classmate's) cuts the identical 30-row test set; with this seed, 43 of the 90 training rows are positive and 17 of the 30 test rows are — close to the file's 50/50 balance, as a random cut should be.

The model is the pipeline from the leakage concept, verbatim: make_pipeline(TfidfVectorizer(max_features=200), LogisticRegression(max_iter=500)). The vectoriser learns the vocabulary and idf values from the 90 training reviews only — on this small file that vocabulary comes out at just 90 distinct words, so the 200-word cap never actually bites here; it is in the code because on any real corpus it must be. max_iter=500 simply gives the optimiser inside LogisticRegression enough iterations to converge cleanly — fitting a logistic model is an iterative search for the best weights, and the default iteration budget sometimes runs out with a convergence warning. model.fit(X_train, y_train) does all the learning in one call, and model.score(X_test, y_test) reports accuracy: the fraction of the 30 held-out reviews routed into the correct queue. Accuracy always deserves a floor to stand on: a lazy 'model' that ignores the text and guesses positive every time would get the 17 positive test rows right and the 13 negative ones wrong, scoring 17/30 ≈ 0.57 — so anything the real model earns must be read against that, not against zero.

Run it, and the score prints 1.000 — all 30 test reviews correct. Resist the celebration: this does not mean sentiment analysis is solved; it means the plumbing works and the file is easy. A templated dataset whose positive rows all contain words like "great" and "amazing" is exactly the kind of corpus a TF-IDF baseline aces; real reviews — with sarcasm, negation like "not good", and misspellings — land such baselines in the 80s and low 90s, not at 100. A more instructive experiment than admiring the 1.000 is to starve the model: change max_features to 20 and the score drops to 0.867; at 10 it is 0.733; at 5 it is 0.467 — no better than flipping a coin. With only five word-slots to see through, most reviews contain none of the surviving words, vectorise to all zeros, and become indistinguishable. That collapse, reproduced in two keystrokes, is the whole argument for giving a text model enough vocabulary to work with.

The last six lines of the starter open the model up instead of just scoring it. model.named_steps lets you reach the fitted pieces inside the pipeline by name — sklearn names each step after its class, lowercased, so "tfidfvectorizer" and "logisticregression" fetch the two stages. From the vectoriser, get_feature_names_out() returns the learned vocabulary in slot order — the fixed word list the whole matrix is counted against, exactly as in the bag-of-words concept, except now it was learned from the training rows rather than written by hand. From the classifier, clf.coef_[0] holds the learned weight for each slot: the same one-number-per-word coefficients you hand-computed with in the linear-classifier concept, now 90 of them, learned from data (the [0] is there because sklearn stores the weights as a matrix with one row per class boundary, and a binary problem has exactly one row). Zipping weights with words and sorting the pairs puts the most negative words first and the most positive last, so printing both ends shows the model's mind: expect broke, quickly, terrible and disappointed among the most negative, and value, amazing and recommend among the most positive — angry words negative, delighted words positive, which is the sanity check passing.

You will also spot something instructive in both lists: filler like "of", "for" and "the" picks up real weight. On 120 templated rows, a filler word that happens to sit inside one class's stock phrasing looks discriminating — "waste of money" puts "of" in negative company often enough for the model to lean on it — and idf, which only zeroes words present in nearly every document, cannot fully police a corpus this small. On production corpora you would counter with min_df (ignore words that appear in too few documents), a stop-word list, or simply more data; and if negation like "not good" is the failure you see, ngram_range=(1, 2) on the vectoriser adds two-word tokens so the phrase gets its own slot and its own weight. Every one of those fixes starts the same way: you printed the coefficients and read them. That habit — score first, then look inside — is the transferable skill this lab is actually teaching, and it survives long after TF-IDF itself has been replaced by whatever your next course reaches for.

What a successful run prints, in words: first the split sizes (90 train, 30 test), then score 1.000, then the five most negative words led by broke and terrible, then the five most positive led by value and amazing.

Coding lab. Sentiment TF-IDF runs in the app, with checks on your output.