PCA

1A table too wide to see

Imagine you wear a fitness tracker for 400 days, and every day it records 20 different numbers: steps walked, calories burned, hours slept, resting heart rate, active minutes, and fifteen more. Written down, that is a table with 400 rows (one row per day) and 20 columns (one column per measurement). Now you want to actually look at your year — do workout days form a cluster? Are there odd days that sit far from the rest? Here is the problem: a chart on a screen has two axes, an x-axis and a y-axis, so you can plot two columns at a time. You cannot plot twenty. The table is simply too wide to see.

But notice something about those 20 columns: they do not move independently. On a heavy-workout day, steps go up, active minutes go up, and calories burned go up — all together, in one coordinated swing. On a lazy day they all sink together. So even though the table has 20 columns, the number of genuinely different stories it tells is much smaller — maybe two or three. Principal Component Analysis, or PCA, is the tool that finds those underlying stories automatically. That is the whole idea: PCA hunts for the few combined directions along which your rows actually differ, so you can keep those and drop the rest.

Figure. The tracker year is a 400-row, 20-column table, but a chart on a screen offers only two axes. Because the columns rise and fall together, the days genuinely differ along only two or three combined directions - and PCA is the tool that finds those automatically so you can keep them and drop the rest.

2Two words: component and variance

Two words in that sentence need grounding before anything else. A direction (PCA calls it a component) is a recipe that blends all 20 columns into one new score — something like 0.4 \times \text{steps} + 0.3 \times \text{active minutes} + 0.2 \times \text{calories} + \dots, giving each day a single number. In the tracker story, that first recipe would read roughly as "how big a workout day was this?" — one column that summarises many. And variance is just the statistician's word for spread: how far the numbers sit from their average. A column where every day is nearly identical has low variance; a column where days differ wildly has high variance. So "the direction with the most variance" means: the blend of columns along which your 400 days differ from each other the most.

3Ranked directions you can add up

PCA hands you its components in a strict order. The first principal component, written PC1, is the single direction that captures the most spread in the data. PC2 is the direction with the next-most spread, chosen at right angles to PC1 — the technical word is orthogonal, and it matters because a right-angle direction cannot repeat any of the spread PC1 already captured. Each component carries genuinely new information, with no double-counting. PCA also tells you how much each component captured, as a fraction called the explained variance ratio: if PC1 reports 0.62, it alone holds 62% of all the spread in your 20 columns, and because components never double-count, you may simply add the fractions — PC1 and PC2 at 0.62 + 0.28 = 0.90 means two numbers per day preserve 90% of how your days varied.

That ordering is exactly what makes PCA useful, in two ways. First, plotting: give every day its PC1 score as the x-coordinate and its PC2 score as the y-coordinate, and suddenly your unplottable 20-column year is a 2D scatter — each dot one day, days that lived similarly sitting near each other. Second, compression: instead of feeding all 20 columns into some later model, keep just the top few component scores. Fewer columns, most of the spread retained.

Figure. Each component reports the fraction of total spread it captured. Because components are orthogonal and never double-count, the fractions add: PC1 at 0.62 plus PC2 at 0.28 means two numbers per day preserve 90% of how the days varied, leaving 0.10 to all the later components combined.

1. Scale the columnsStandardise features so one large unit does not dominate the variance directions.
2. Fit on train onlyLearn components from X_train; transform test with the same rotation — never refit on test.
3. Keep enough varianceChoose the number of components by cumulative explained variance, not by eye on a scatter alone.

4Spread is not signal

One warning, because it is the classic trap. PCA never looks at labels. If each day were tagged "felt great" or "felt tired", PCA would not know and would not care — it is unsupervised, meaning it works from the measurements alone. It ranks directions purely by spread, and spread is not the same thing as usefulness for prediction: a direction with tiny variance can still be exactly the axis that separates "felt great" from "felt tired" days, and dropping it because it looked small can quietly throw away the signal a classifier needed. PCA is a summariser, not a predictor. One practical corollary: because PCA chases variance and variance follows units, always scale your columns first — otherwise a column measured in big numbers (steps, in the thousands) would dominate every component while sleep hours (around 7) barely registered, purely because of units.

The picture below shows the whole idea shrunk to just two columns, so it fits on a page: each dot is one day, plotted by steps walked against calories burned (both scaled). The dots form a stretched, tilted cloud, because the two columns rise and fall together. The solid line through the cloud's long axis is PC1 — the direction of most spread. The short dashed line at right angles is PC2, holding what little spread remains. Keep only each day's position along PC1 and you have compressed two columns into one while losing the least information. With your real table, the same thing happens in 20 dimensions — you just cannot draw that, which is precisely why PCA is the tool that lets you see it.

Figure. Each dot is one day, plotted by steps walked against calories burned. The cloud is elongated because the two columns rise and fall together, so most of its spread lies along one direction: PC 1. The dashed PC 2, at right angles, carries what little variance is left. Keeping only the PC 1 coordinate compresses two columns into one while losing the least spread - that is the whole trade, and no class labels were consulted.

5Lab: PCA projection

Time to run PCA for real. Since you do not have 400 days of tracker data lying around, Cell 1 manufactures a stand-in: make_classification builds a synthetic table X with 400 rows and 20 numeric columns — same shape as the tracker year — plus an array called labels that tags each row with one of three classes (think of them as "rest day", "light day", "training day"). Keep in mind what each name holds: X is the wide table of measurements, and labels is the tag per row that PCA itself will never be shown.

Cell 1 then does two things in order, and the order matters. First, StandardScaler().fit_transform(X) rescales every column to the same footing — mean 0, spread 1 — producing X_scaled. This is the scale-first rule from the concept above: PCA ranks directions by variance, and variance follows units, so without this step whichever column happened to use the biggest numbers would win PC1 by default. Second, PCA(n_components=2) asks for exactly the top two components, and fit_transform hands back X_pca: still 400 rows, but now only 2 columns — each row's PC1 score and PC2 score. Twenty columns in, two out.

The line worth reading closely is the print of explained_variance_ratio_. It shows two fractions, one per kept component — say [0.31, 0.12]. The first means PC1 alone captured 31% of all the spread in the 20 scaled columns; the second means PC2 added another 12%. Add them and you know your 2D picture preserves 43% of how the rows varied — and, just as usefully, that 57% of the spread lives in the 18 directions you threw away. The notebook cells share one kernel, so X_pca and labels stay alive after Cell 1 finishes; Cell 2 does nothing but draw, scattering each row at its (PC1, PC2) coordinates and colouring each dot by its entry in labels.

Be precise about what that colour means, because this is where beginners over-read the plot. PCA was fitted on X_scaled only — it never saw labels. The colouring is applied afterwards, purely so your eye can check whether the classes happen to land in different regions. If they do, that is a pleasant discovery, not something PCA aimed for. And the reverse misreading matters too: a high explained variance ratio does not promise separated colours. The ratio measures how much of the input spread survived the compression — nothing about classes. Two components can keep 85% of the spread while the colours overlap completely, because the direction that distinguished the classes was one of the small ones PCA ranked low and you dropped.

So if your scatter looks like one undifferentiated blob, do not conclude the lab failed. It may mean the class structure lives in more than two directions, or in low-variance ones. You can try more informative features, keep more components, or simply accept the honest answer: two components did not carry the structure you cared about — which is itself something you now know about your data.

No diagram — the projection is drawn by the coding lab plot, not a static figure.

Coding lab. Scale, PCA to 2D, scatter runs in the app, with checks on your output.