Machine Learning · Machine Learning Core

Learning Setup

Learning types, problem framing, and feature/label layout are the contract before any classical model — hold-out splits land in the next lesson.

Before any model code, lock two things: which learning family fits the signal you hold, and what exact event the label names — with features known at predict time. Getting those wrong makes every later score look better than the model really is. The concrete train/test cut is the next lesson.

Machine Learning
Medium level
7 concepts
11 practice questions

1Learning from an answer sheet

Start with something you can picture: a notebook listing two hundred flats your office has already rented out. For each flat the notebook records a few plain facts — the size in square feet, the distance from the city centre in kilometres, the age of the building in years — and, on the same line, the answer we care about: the monthly rent a tenant actually agreed to pay. Every line is a solved example, with the facts and the correct answer sitting side by side.

When your data comes as question-and-answer pairs like this, teaching a computer from it is called supervised learning. The recorded facts are the inputs, and the known correct answer — here, the rent — is called the label. The word 'supervised' simply means an answer sheet exists: the computer makes a guess for each flat, compares its guess against the real rent written on that line, and corrects itself. Spam filtering is the same setup (every training email is marked spam or not spam by a person), and so is loan scoring (every past loan is marked defaulted or repaid by history).

Figure. Supervised learning in one loop: the model reads each flat's recorded facts, guesses a rent, compares the guess against the real rent written on the same line, and corrects itself. The known answer on every line - the label - is what makes the learning supervised.

One round of the supervised loop, by hand

Take the three flats already in the notebook grid: 850 sq ft, 4.0 km from the centre, 12 years old, rent 18; 1,100 sq ft, 1.5 km, 5 years, rent 30; 600 sq ft, 9.0 km, 20 years, rent 11 — rents in thousands of rupees per month. Play the model for one round with a deliberately crude first rule: guess the rent as size times 0.02, which is Rs 20 per square foot (0.02 rather than 20 because the rents are recorded in thousands). Run the supervised loop by hand: make a guess for each flat, compare each guess against the real rent written on the same line, and read off what the answer sheet tells the rule to fix. Then tear the rent column out and try to run the same loop again.

Guess for the first flat, size times the rate: 850 × 0.0217 — against a real rent of 18
Guess for the second flat: 1,100 × 0.0222 — against a real rent of 30
Guess for the third flat: 600 × 0.0212 — against a real rent of 11
Compare, line by line: each error is the real rent minus the guess, so 18 − 17, then 30 − 22, then 11 − 12+1, +8, −1
Read what the answer sheet just said. The first and third flats are nearly right, but the second flat — the one only 1.5 km from the centre — is under-guessed by 8. A rule that reads only size cannot see closeness to the centre, and the labels have pointed at exactly that blind spotthe +8 miss names the missing feature
Now tear out the rent column and rerun the loop. The guessing step still works — 850 × 0.02 is still 17 — but the comparing step has nothing to subtract: 17 minus what?no labels, no errors, no correction — the loop halts

Pro tip. The label column is not there for the model to memorise — it is there to price each mistake. The same crude Rs-20-per-square-foot rule became improvable the instant every line carried a real answer to compare against; 'supervised' describes the data you hold, not the cleverness of the algorithm.

A hospital records patient vitals and wants to predict the exact number of days until discharge. Is this regression or classification, and why?

Classification, because each patient eventually belongs to one discharged cohort
Unsupervised learning, because recovery times cannot be labelled in advance
Neither, because medical records require reinforcement learning policies
Regression, because length of stay is a continuous count of days rather than a fixed discrete category

When the target you want to predict is a continuous quantity or numeric measurement (like days or rupees), the supervised task is regression. Classification applies when predicting discrete class labels.

2Learning without an answer sheet

Now imagine the rent column torn out of the notebook. You still know every flat's size, distance and age, but there is no answer to compare a guess against, so 'predict the rent' is no longer a job you can train for. What you can still do is look for structure in the inputs alone: perhaps the flats fall into natural groups — small central studios here, large suburban family flats there — that nobody had named in advance. This is unsupervised learning: inputs only, no answer sheet, and the goal is to discover patterns such as clusters, unusual rows, or simpler summaries of the data rather than to reproduce a known answer.

The third family, reinforcement learning, has neither an answer sheet nor a bare pile of inputs. A program acts — moves a game piece, adjusts a thermostat — and only afterwards receives a reward or a penalty, from which it slowly learns which actions earn more reward. It is how machines learn to play games. This course stays with the first two families: mostly supervised work, plus a short unsupervised slice — not agents and rewards.

You hold a year of server logs with no incident labels and want to surface unusual days. Which family fits?

Supervised, using the date of each record as the label
Unsupervised, because you have inputs and nothing to predict them against
Reinforcement, because the server reacts to what you do to it
None: unlabelled data cannot be modelled

The signal you hold picks the family. No target column means nothing to supervise, so the job is finding structure rather than reproducing an answer.

3The signal picks the family

So when a new problem lands on your desk, the deciding question is not which library you prefer. It is: what signal does my data hold? Facts paired with known answers mean supervised learning. Facts alone mean unsupervised learning. Rewards arriving after actions mean reinforcement learning. The signal you hold picks the learning family — not the library you prefer.

Figure. The signal you hold picks the family: labelled pairs for supervised, inputs alone for unsupervised, rewards after actions for reinforcement. This course lives in the left two panes - supervised work plus a short unsupervised slice - and leaves the agent loop for another day.

Learning types
Type	Signal you hold	Example job
Supervised	Input → label pairs	Predict churn from last-90-day usage
Unsupervised	Inputs only	Group customers with similar baskets
Reinforcement	Rewards from actions	Learn a game policy by playing

You have 50,000 unlabelled customer purchase receipts and want to discover natural shopping pattern clusters. Which family applies?

Supervised regression, using total basket spend as the dependent label
Unsupervised learning, because there are no ground-truth group labels to train a classifier against
Reinforcement learning, by awarding bonus points to frequent shoppers
Supervised classification, by arbitrarily numbering clusters 1 through 5

Without pre-existing target labels, the model searches for geometric structure or groupings directly from features, which is the definition of unsupervised learning.

4Frame the prediction problem

'Predict the rent of a flat' sounds like a complete instruction, but it hides three unanswered questions, and different answers to them produce genuinely different projects. So before you touch a model, write three lines on paper: the decision the prediction will change, the exact definition of the label, and the time at which each input is known. Each line closes one of the hidden questions.

First line — the decision. Ask: when the model produces its number, what will someone do differently? For the rent model the answer might be 'we set the advertised price of each new listing'. If the honest answer is 'nothing would change', stop there: the model is decoration, and no accuracy score can fix that.

Second line — the label, meaning the exact real-world event the answer column records. This is where two teams who both say 'customer churn' can quietly build different products: 'customer left within 30 days' and 'customer left within a year' are different events, so the two models answer different questions and their scores cannot be compared. The rent version of the same trap: 'the rent we advertised' and 'the rent the tenant finally signed at' are different numbers — pick one, in writing, before any code exists.

Framing checklist
Lock this	Write it as
Decision	What changes if the score is high?
Label	Which event, over which window?
Feature time	Known before the decision, for every row?

Two teams each build a 'customer churn' model and cannot reconcile their results. Which framing choice most likely explains it?

One team worked in Python and the other in R
One team held out a larger test split than the other
They counted churn over different windows, so the two models are predicting two different events
One team scaled the features and the other did not

The label definition is part of the problem, not an implementation detail. Left off in a month and left off in a year are different questions and deserve different models.

5Feature timing and leakage

Third line — feature timing. Every input must be knowable at the moment the prediction is made. A column that only gets filled in after the event has effectively smuggled the answer into the inputs; this is called leakage. For example, 'discount the tenant negotiated' only exists once a tenant has already signed — a rent model trained with that column looks brilliant in the lab, because it is partly reading the answer, and then collapses in production, where the column is still empty at prediction time.

This is why vague framing is more dangerous than a weak algorithm: a vaguely framed model can score beautifully in a notebook and still fail the moment someone has to act on its prediction. The three written lines cost five minutes and prevent that entire class of failure.

No diagram — the idea is carried by the prose, table or coding lab.

Name the decisionBefore choosing any algorithm, write down the action the prediction will change — set a listing price, approve a loan, call a customer back. A prediction nobody acts on is not worth modelling.
Lock the labelDefine the answer column (the label, usually written y) precisely: which event it records, for which row, over which time window, and at what moment its true value becomes known.
Check feature timingWalk through every input column (the features, collectively called X) and ask: is this value already known at the moment we predict? Drop anything recorded after the decision — that is leakage, not information.

Timing audit of the notebook columns
Column	When its value exists	Verdict
size, distance, age	Before any tenant — a tape measure and the building record give all three	Keep: known for a still-empty flat
advertised_rent	Only once we set the listing price — the very decision this model exists to change	Drop: the decision's output cannot be its input
discount_negotiated	Only after a tenant has haggled and signed — after the label event	Drop: leakage, not information
advertised_rent minus discount	20 - 2 = 18, 32 - 2 = 30, 12 - 1 = 11 — the signed rents exactly	The near-zero lab error was leakage, not skill

Auditing a too-good rent model

A junior analyst extends the rental notebook. For the three flats already in the grid — 850 sq ft, 4.0 km, 12 years; 1,100 sq ft, 1.5 km, 5 years; 600 sq ft, 9.0 km, 20 years — she adds two new columns: advertised_rent (20, 32, 12, in thousands of rupees) and discount_negotiated (2, 2, 1). The label y is the signed rent: 18, 30 and 11. Her retrained model's lab error drops to almost exactly zero. Should the two new columns ship?

Run every column through the timing question — would I already know this value for a brand-new, still-empty flat? Size (850 sq ft), distance (4.0 km) and age (12 years): a tape measure and the building record give all three before any tenant appearsthe three original columns stay
advertised_rent, the sneaky one: the decision this model exists to change is 'we set the advertised price of each new listing'. At prediction time that price has not been decided yet — it is the output of the decision, so it cannot be an input to itfails the timing test
discount_negotiated, the plainer one: a discount exists only after a tenant has haggled and signed, which is after the label event. A column filled at or after the event is leakage, not informationfails the timing test
Now the near-zero lab error explains itself. On every row, advertised rent minus discount equals the signed rent exactly — 20 - 2 = 18, 32 - 2 = 30, 12 - 1 = 11 — so the model was subtracting one answer-bearing column from another, not predicting rent from the flat's factsthe zero error was leakage, not skill
Drop both columns and retrain on size, distance and age alone. The lab error rises — and the larger number is the honest one, because it is measured on inputs production will actually have when the flat is still emptydo not ship the two new columns

Pro tip. An error that looks too good to be true usually is. Before celebrating, ask when each column gets its value: anything filled at or after the label event is the answer wearing a feature's name.

A churn prediction model includes 'total calls to cancellation hotline this month' as an input. Why is this feature problematic?

It is a categorical string rather than a numeric floating-point value
It has zero correlation with whether the customer leaves the service
It forces the model to use logistic regression instead of decision trees
It introduces data leakage, because calling the cancellation line occurs after the customer has already resolved to churn

A feature that incorporates future actions or events that only happen once the outcome is underway leaks the label into training, causing unrealistically high test scores that fail in production.

6Features and labels

Stay with the rental notebook, and take just three flats that already found tenants. For each flat we know three facts — its size in square feet, its distance from the city centre in kilometres, and the age of its building in years — and we know the answer: the monthly rent the tenant agreed to pay, in thousands of rupees. Each recorded fact is called a feature, and the known answer is called the label. Nothing about a flat that we did not write down exists as far as the model is concerned.

A computer cannot leaf through a notebook, so we arrange the same information as a grid: one row per flat, one column per fact. That is exactly what the diagram shows. Read the first row left to right — 850, 4.0, 12 — and it is one flat: 850 square feet, 4 km from the centre, 12 years old. The second row is a different flat: 1,100 square feet, 1.5 km out, 5 years old. So a row is one flat whose rent we already know, and a column is one kind of fact recorded for every flat — the size column, the distance column, the age column.

This grid of inputs is called the feature matrix and is written X — capital, because it holds many rows and columns at once. The answers sit in their own single column, written y: here 18, 30 and 11, each flat's actual monthly rent in thousands of rupees, listed in the same order as the rows of X, so row two of X and entry two of y describe the same flat. And when textbooks write x1, x2, x3, they are doing nothing deeper than numbering the columns: x1 is column one (size), x2 is column two (distance), x3 is column three (age). The whole point of this layout is that the computer can study many solved examples at once — run down a column to compare every flat's size, run across a row to see one flat whole, and notice that bigger, closer, newer flats sit beside larger numbers in y.

Figure. One row is one flat we already rented out; one column is one fact recorded about every flat. The X block holds what the model can see at predict time - size in square feet, distance from the centre in km, building age in years - and the separate y column holds the answer it must learn to reproduce: the monthly rent each flat actually fetched, in thousands. A column only knowable after the rent is agreed is leakage, not a feature.

In a flat rental pricing dataset with columns [sqft, bedrooms, metro_dist_km, rent_rupees], what role does rent_rupees play?

An input feature x used to estimate the flat's square footage
The supervisory target label y that the model attempts to predict from the feature vector x
A regularisation hyperparameter that scales the loss penalty
A one-hot encoded category indicating tenant income bracket

The column representing the outcome to be predicted is the ground-truth target label y, while the preceding descriptive columns form the input feature vector x.

7Encoding and the timing test

One thing the grid cannot fix by itself: a model reads every column as a quantity. That is fine for size and distance — 1,100 square feet genuinely is more than 850. But a city name is not a magnitude: if a listings dataset maps Pune, Kochi and Surat to 1, 2 and 3, a linear model will treat Surat as 'three times Pune' and place Kochi exactly halfway between them — an order and a spacing the data never had.

So categories get encoded rather than numbered. One-hot encoding gives each city its own column holding 1 for 'this flat is in that city' and 0 otherwise, so no fake ordering sneaks in. Continuous columns like size and distance are often rescaled to comparable ranges when the model cares about relative magnitudes across features.

Finally, put every column through the timing question from the previous concept: would I already know this value for a brand-new, still-empty flat? Size, distance and age — yes. 'Discount the tenant negotiated' — no, that only exists after the rent is settled, so it belongs with the answer, not the inputs. A column that arrives only after the label event is leakage, not a clever feature.

Column roles
Piece	Holds	Common mistake
X	Inputs known at predict time	Including a column filled after the event
y	The label you defined	Mixing two label windows in one column
Category col	Names / codes	Integer-coding cities for a linear model

One-hot encoding the city column, by hand

The three notebook flats gain a city column: the 850 sq ft flat is in Pune, the 1,100 sq ft flat is in Kochi, and the 600 sq ft flat is in Surat. A colleague proposes the quick fix — map Pune to 1, Kochi to 2, Surat to 3 and keep a single numeric column. First run the arithmetic a linear model would actually do with that column, to see what the mapping forces; then build the one-hot columns by hand and count the shape of the new grid.

A linear model multiplies a numeric column by one weight. Suppose the fit lands on a city weight of Rs 400 per month per code unit. Each flat's city contribution is its code times that weight: Pune 1 × 400, Kochi 2 × 400, Surat 3 × 400Rs 400, Rs 800, Rs 1,200
Read what those three numbers force: Surat's contribution is 1,200 ÷ 400 = exactly 3 times Pune's, and Kochi's 800 sits exactly halfway between 400 and 1,200. No fitted weight can undo this — with any weight w, the contributions are w, 2w and 3w, equally spaced, in code orderthe fake ordering lives in the code, not in the fit
One-hot instead: replace the single city column with three columns — is_pune, is_kochi, is_surat — each holding 1 for 'this flat is in that city' and 0 otherwise. The Pune flat's three entries1, 0, 0
The Kochi flat and the Surat flat, same rule0, 1, 0 and 0, 0, 1
Sanity-check each row: a flat is in exactly one city, so every row's three city entries add to 1 + 0 + 01, on every row
Count the new grid: the three original fact columns (size, distance, age) plus the three city columns, so 3 + 3 columns for the same 3 flatsX is now 3 rows × 6 columns
Why the extra columns pay: each city column gets its own weight. Suppose the fit now lands on Rs 500 for is_pune, Rs 900 for is_kochi and Rs 300 for is_surat. The Kochi flat's city contribution is 0 × 500 + 1 × 900 + 0 × 300Rs 900 — and Kochi (900) now exceeds Surat (300), an ordering the integer code could never produce

Pro tip. One-hot encoding does not add information — it removes a false claim. The integer code chains all three city effects to a single number, so they always land equally spaced in code order; three one-hot columns give each city its own weight, and the data — not your labelling order — decides which city commands a premium.

A city column holds 'Pune', 'Kochi' and 'Surat'. You map them to 1, 2, 3 and fit a linear model. What have you told the model?

Nothing it cannot cope with, since the mapping is arbitrary and the fit ignores it
That the cities lie in that order on one axis and that the third is three times the first, neither of which is a fact about cities
That the three cities are mutually exclusive categories
That city is the target rather than one of the features

A linear model reads a column as a magnitude. Integer codes for names smuggle in an order and a spacing that the data never had, which is why categories are encoded rather than numbered.

Notes

Supervised vs unsupervised signal, problem framing, and feature/label layout.
The signal you hold picks the learning family — not the library you prefer.
Write three lines before you touch a model: the decision the prediction will change, the exact label definition, and the time each feature is known.

Exam traps & shortcuts

Keep lab datasets under 2000 rows in the browser runtime.
Split train and test before fitting any model that sees labels.

Recap

This lesson in brief:

Learning types: The signal you hold picks the learning family — not the library you prefer.
Frame the prediction problem: Write three lines before you touch a model: the decision the prediction will change, the exact label definition, and the time each feature is known.
Features and labels: In the usual matrix layout, X holds features (one row per example, one column per input) and y holds the target the model must predict.

Practise Learning Setup

Reading is free and needs no account. Practice, mocks and progress live in the app.

11 exam-style questions on this topic, with explanations
A 5-question practice set that ends the chapter
Timed mocks scored with the real marking scheme
Readiness tracked per topic, kept on your device

Continue with Google — freeNo card, no trial. Works offline once installed.