Customer Churn Application

1The business question

Picture the retention desk at a telecom company. Every month some subscribers quietly leave — they stop paying and never call to say goodbye — and the business word for leaving is churn. The desk can fight back with a save offer (a discount, an upgrade, a call from a human), but each offer costs money and the team can only work through a limited list. Meet Anita, one subscriber on this book: she joined six months ago, is on a month-to-month contract that lets her walk away any billing cycle, pays a mid-range bill, and has phoned support twice. Nothing in that description says 'leaving' outright — the whole job of this lesson is deciding, from numbers like hers, who belongs on the outreach list before they go.

So write the decision down before any model exists: which customers get a save offer this month? A machine-learning model helps by giving every subscriber a churn score — a number saying how likely that account is to leave — and the useful product is the ranked list those scores put in order, with the limited outreach budget spent from the top down. Hold on to that framing, because it decides everything later in the lesson: what the churned label must mean, which columns are fair to use as inputs, and why a single accuracy percentage is never the campaign.

No diagram — the idea is carried by the prose, table or coding lab.

2Lock the churn window

Before a single row of training data can exist, someone must turn the word churned into a yes-or-no column, and that takes two decisions. First, pick a snapshot date — the day you stand on and look forward from; everything the model may know about an account is what was already true on that day. Second, pick a window — how far past the snapshot you watch before scoring the answer. 'Churned' then means exactly one thing: the account went silent inside the window. Change the window and you have changed the question.

See it bite on one customer. Suppose Anita keeps paying for a while after the snapshot and finally goes silent around day 100. A team that locked a 30-day window looks at day 30, sees her still paying, and writes churned = 0 — she counts as a stay. A team that locked a 365-day window looks anywhere inside the year, sees her leave at day 100, and writes churned = 1. Same customer, same history, opposite labels. Two models built by those two teams are answering different questions, so their accuracy numbers cannot be compared, however precise both sound.

The teaching table this lesson uses locked the wide lane: its churned column means the account went silent within the year after the snapshot, which is why Anita's row carries a 1. Whatever window your own project locks, write it down before any code exists — the label definition is a business decision the model inherits, not something the fit discovers.

Figure. One customer, two locked windows. Anita goes silent around day 100 after the snapshot: past the 30-day lane's cut, so that team writes churned = 0, but inside the 365-day lane, so that team writes churned = 1. The two teams' models predict different events and their scores cannot be compared.

3Signals known before the window opens

With the label locked, choose the features: the columns the model may read when it scores an account, every one of them a fact that was already true on the snapshot date. The teaching table keeps four. tenure_months is how long the account has existed — Anita's row says 6. monthly_charges is the size of the bill, stored as a plain number — hers is 75.67. support_calls counts calls to the help line — hers is 2. contract_month_to_month is a 1 when the customer can leave any month without penalty and a 0 when they are locked into a longer deal — hers is 1. The fifth column, churned, is the label: the answer the model must learn, never an input it may read.

Do these columns actually carry signal? Count, straight from the 200 rows. Among the 96 month-to-month accounts, 26 churned — a rate of about 27%. Among the 104 locked-in accounts, 14 churned — about 13%. One column, and the churn rate roughly doubles. Tenure tells the same kind of story: churners average about 24 months on the book against 37 for stayers. This is what 'a feature carries signal' means in practice — knowing the column's value genuinely shifts how worried you should be about the account.

One more test before a column is allowed in, and it is the test real projects fail: was the value knowable on the snapshot date? A column like called_by_retention looks like a superb predictor — accounts flagged in it churn heavily — but it records something the company did after it already suspected the churn. Training on it is called leakage: the model looks brilliant on historical data and useless on deployment day, because at prediction time that future column is still empty.

Figure. Churn rate by contract type, counted from the teaching table's 200 rows: 26 of 96 month-to-month accounts churned (27.1%) against 14 of 104 locked-in accounts (13.5%). The dashed rule is the whole-table rate, 40 of 200 = 20%.

1. Freeze the snapshotFix the date you predict from. Features describe the world on or before it; the label describes the window after it.
2. Interrogate each columnAsk of every candidate feature: was this value already known on the snapshot date, for every account?
3. Drop what failsColumns written after the snapshot — like called_by_retention — leak the outcome and must go, however predictive they look offline.

4Pick a metric

Now the trap this topic is famous for. Churn is a minority event — in the teaching table only 40 of 200 accounts churned, a base rate of 20% — so the two classes are badly unbalanced, and that imbalance breaks accuracy as a report card. A lazy model answering 'stays' for every account is correct on all 160 stayers and wrong on the 40 churners: 80% accurate, zero leavers caught. On the lab's held-out test fold the same trick scores 90%, because that fold happens to hold just 5 churners among its 50 rows. An accuracy number can be high precisely because the model ignores the customers you built it to find.

So grade the model on the minority class itself. Recall of churners asks: of the customers who truly left, what fraction did the model flag? Precision asks: of the customers the model flagged, what fraction truly left? A save-offer campaign with spare capacity favours recall — missing a leaver costs a whole customer, while a wasted offer costs about ₹500. A campaign that annoys the people it contacts favours precision. And when the budget is a fixed list, the cleanest habit is to measure both at the top of the ranking — the slice of the list the team will actually work through.

Figure. On the lab's 50-row test fold with 5 churners, the do-nothing model and the fitted model print the same 90% accuracy — the first pair of bars is identical. The recall bars tell them apart: 0 of 5 churners against 1 of 5. Accuracy hides exactly the difference that matters.

1. Count churn rateIf only 8% leave, a 'never churn' model can look 92% accurate and catch nobody.
2. Pick recall or precisionSave offers favour catching leavers (recall); spammy campaigns favour precision.
3. Report on hold-outFit on train, score precision/recall on test — the fold the fit never saw.

5From scores to a call list

The fitted model does not have to answer yes or no. Ask it for predict_proba and it returns, for every account, a churn score between 0 and 1 — its estimate of the probability that the account's churned label is 1. Anita's four numbers, pushed through the lab's fitted model, come out at about 0.39. Read against the table's 20% base rate, that is a loud warning — roughly twice the average risk — even though a yes/no model with the usual 0.5 cutoff would file her under 'stays'. Sorting every account by this score, highest first, is what turns a classifier into the ranked call list the business question asked for.

Ranking also survives the arithmetic of money. A save offer costs about ₹500; a retained subscriber is worth about ₹3,000 in future billing — her customer lifetime value, or CLV. Calling Anita therefore risks ₹500 to protect an expected 0.39 × ₹3,000 ≈ ₹1,170, worth doing even though her score sits below one half. The 0.5 cutoff is a default of the software, not a law of retention: the sensible threshold falls out of capacity and cost, and with a ranked list you often need no threshold at all — the team simply works from the top of the list until the budget runs out.

Figure. Churners reached by 10 calls into the 50-account test fold. Calling at random reaches about 1 of the 5 churners (10 calls × 10% churner share); calling the top 10 accounts by model score reaches 4 of the 5. The dashed line marks the ceiling — all 5 churners hiding in the fold.

6Lab: churn model

Time to run the whole lesson end to end on the real teaching table — the same 200 accounts every figure above was counted from, Anita's row included. This is a three-cell notebook sharing one running Python session (the kernel): Cell 1 fits the model and prints the famous score, Cell 2 looks past that score at churners actually caught, and Cell 3 draws what the model learned. Run them in order — the model and test fold built in Cell 1 stay alive for the later cells — and leave the CSV at 200 rows, because the notebook runs in your browser and small tables keep it instant.

Cell 1, line by line. pd.read_csv loads the CSV into a DataFrame — a table in code. features names the four columns the model may read, and X, y = df[features], df["churned"] cuts the table into X, the 200 × 4 grid of inputs, and y, the 0-or-1 answer column. train_test_split(X, y, test_size=0.25, random_state=0) holds out a quarter of the rows — 150 accounts to fit on, 50 the fit never sees — and pinning random_state means every run, and every classmate, gets the identical cut. LogisticRegression(max_iter=500) builds the classifier (max_iter simply gives its optimiser enough steps to settle); model.fit(X_train, y_train) learns one weight per feature plus an intercept; and model.score(X_test, y_test) prints the held-out accuracy. Expect score 0.9.

Cell 2 is where the topic's lesson lands. The test fold holds 5 churners among its 50 accounts, so a model answering 'stays' for everyone also scores 0.9 — the printed accuracy sits exactly on the always-stay baseline and by itself proves nothing. The cell prints two better numbers from the same fitted model: at the default 0.5 cutoff it flags only 1 of the 5 churners (recall 20%), yet the top 10 accounts ranked by churn score contain 4 of the 5. That pair of numbers is the whole business case in miniature — as a yes/no oracle this small model is feeble, but as a ranking engine for a 10-call budget it concentrates churners four times better than random outreach.

Cell 3 bars the fitted weights, one per feature, and their signs should now read like the lesson. tenure_months carries a negative weight — long-standing accounts leave less. contract_month_to_month carries the largest positive weight: the flexible contract is the risk flag, the 27% versus 13% you counted earlier seen through the model's eyes. support_calls adds a smaller positive push, and monthly_charges barely moves the score on this sample. Two habits to take away: keep random_state pinned so every re-run stays comparable, and treat X_test and y_test as frozen — every number you quote about the model must come from rows the fit never met.

No diagram — the figure is Cell 3's coefficient bar chart, drawn from the model you just fitted.

Coding lab. Churn baseline, then look past the score runs in the app, with checks on your output.