Credit Risk Application

1Frame the decision

A lender wants a default-risk score before issuing credit — the prediction changes approve / review / decline, not a slideshow. Meet the applicant this whole lesson follows: Meera, 27 years old, asking for a $12,000 personal loan repaid over two years. The bank knows four facts about her on the day she applies — her income ($48,000 a year), her debt ratio (0.52, meaning 52 cents of every dollar she earns is already committed to existing repayments), her record of late payments (2 in the last two years), and her age. The model's entire job is to turn those four numbers into one: P(\text{default}), the probability that Meera fails to repay.

Before touching any data, put a price on the two ways the bank can be wrong about her. Approve Meera and have her default, and the bank loses the money it lent — up to the $12,000 principal. Decline Meera when she would have repaid, and the bank loses only what it would have earned — about $1,400 of interest over the two years — plus a wronged applicant and fairness scrutiny. The two mistakes do not cost the same, and every later choice in this lesson — which metric to headline, where an approval threshold sits — inherits that asymmetry.

Figure. The price of each mistake on Meera's loan: approving a defaulter risks the 12,000 of principal, while declining a good borrower forfeits about 1,400 of margin. One missed default erases the margin of roughly nine good loans, which is why nothing downstream may treat the two errors alike.

2Lock the label and the clock

What exactly is the model predicting? "Default" sounds obvious until you have to write it down. Lock a label window before any code — a working definition such as: defaulted = 1 means the borrower fell 90 or more days behind on payments within 12 months of the loan being issued. Change the window — 6 months instead of 12, 30 days behind instead of 90 — and the same borrower can flip from 1 to 0, so two models trained on different windows are answering different questions and their scores cannot be compared. If those lines are vague, every later ROC number is theatre.

The second lock is on the clock. Every feature must be a fact the bank can know on the day Meera applies. Her income, her debt ratio, her past late payments, her age — all knowable at application time, all legitimate. But a column like "instalments missed on this loan so far" only comes into existence after the money is out the door. Train on it and the model looks brilliant — it is effectively reading the answer off the sheet — and then collapses in production, where that column is blank at the moment of decision. This is leakage: the future seeping into features that are supposed to describe the present.

Figure. The timeline that separates legal features from leakage: everything the model may use is frozen on application day, before the money moves; the defaulted label only comes into existence at the end of the 12-month window. A feature drawn from anywhere right of the decision is reading the answer.

1. State the decisionThe score changes approve, review, or decline — not a marketing segment.
2. Define defaultLock what 'defaulted=1' means: days late, write-off, or charge-off — and the window it is measured in — before loading the CSV.
3. Check feature timingIncome and debt ratio must be known at application — drop any column that only exists after disbursement.

3Inspect the sample

The bundled credit_risk_v1.csv carries income, debt ratio, late payments, age, and a binary defaulted label. It holds 200 rows, one per past applicant whose outcome is already known — which is exactly what supervised learning needs: the answers are on the sheet. If Meera were a row, she would read 48000, 0.52, 2, 27, plus the 0 or 1 that only her 12-month window can write. Across the file, incomes run from about $25,000 to $140,000, debt ratios from 0.06 to 0.73, late payments from 0 to 6, and ages from 21 to 70. Glance at those ranges before fitting anything: a debt ratio of 7.3 or an age of 3 is caught by eyes, not by sklearn.

Now count the labels: 54 of the 200 applicants defaulted — a 27% default rate. That is deliberately heavy. A real lender's book runs at low single digits — say 4 defaults per 100 loans — but a 200-row teaching sample at 4% would contain just 8 defaulters, far too few to learn anything from in a browser lab. So hold both numbers at once: the sample you practise on is 27% defaults; the book you would ship to looks more like 4%. It is a teaching subset sized for the browser — not a bank production extract, and not a claim about any real portfolio.

Figure. The 200-row teaching sample: 146 applicants repaid and 54 defaulted, a 27% default rate. The rare class is deliberately over-represented so a browser lab has enough defaulters to learn from; a real lender's book runs at low single digits.

4Score the model like a lender

Accuracy is the friendliest metric — the fraction of all rows labelled correctly — and on this problem it is also the most misleading one. Consider the laziest possible model: it approves everyone, predicting "repays" for every applicant, and never catches a single defaulter. On the 200-row sample it gets the 146 good borrowers right and all 54 defaulters wrong: accuracy 146 ÷ 200 = 0.73, earned while doing literally nothing a lender needs. On a real book at a 4% default rate the same do-nothing model scores 0.96 — the rarer the defaults, the more flattering accuracy becomes.

So headline the metric that asks the lender's question. Recall on the default class: of the applicants who truly defaulted, what fraction did the model flag? The do-nothing model flags none of them, so its recall is 0 of 54 = 0.00 — the number that exposes it. Recall's partner is precision: of the applicants the model did flag, what fraction really defaulted — the price being paid in false alarms. And when the fit itself keeps ignoring the rare class, two standard levers push back: class_weight='balanced', which makes each missed defaulter cost the fit more, or oversampling defaults — applied in the training fold only, never in test, or the test fold stops being an honest preview of production.

Figure. The same do-nothing model, scored on two books: on the 27% teaching sample it reads 0.73, and on a real book at 4% defaults it reads 0.96 — while catching zero defaulters on both. The rarer the defaults, the more flattering accuracy becomes, which is why recall on the default class is the headline number here.

5From probability to decision bands

The model hands back a probability; the bank must act on it. The naive rule — flag whenever P(\text{default}) \ge 0.5 — quietly assumes both mistakes cost the same, and the framing concept priced them at $12,000 against $1,400. Expected value does the honest arithmetic instead: approving an applicant whose default probability is p earns about (1-p) \times 1400 of margin and risks p \times 12000 of principal. Approving pays, on average, only while the first number beats the second — and the ledger below shows that stops at p^* \approx 0.104, nowhere near 0.5.

Real credit desks go one step further than a single cutoff: three bands. Below a low cutoff (say 0.08) the loan is auto-approved; above a high one (say 0.25) it is auto-declined; between the two, a human reads the file — the middle band spends reviewer time exactly where the model is least sure. Meera's score of 0.19 lands in that middle band: not rubber-stamped, not refused, but queued for review. Both cutoffs are tuned on validation data against portfolio targets — approval rate, review workload, expected default rate — and never on the test fold, which must stay a sealed preview of production.

Figure. The probability axis cut into three actions: auto-approve below 0.08, manual review from 0.08 to 0.25, auto-decline above 0.25. The dashed rule marks Meera's score of 0.19 — inside the review band, so a human reads her file. Each segment's width is the span of probabilities it covers.

6Lab: default classifier

Load /data/credit_risk_v1.csv, split with a fixed seed, fit LogisticRegression, and print hold-out accuracy via model.score. That is the skeleton — but this lab carries a twist worth the whole lesson: the first fit will print a respectable-looking accuracy while doing nothing at all, and the two refits after it show how to catch that and how to fix it. Every number quoted below is what the code actually prints with random_state=0, so you can check each claim against your own output, line by line.

Walk the top of the script first. pd.read_csv loads the 200 applicants into a table called df. The line X = df[[...]] picks out the four feature columns by name — income, debt_ratio, late_payments, age_years, the facts known on application day — and y = df["defaulted"] is the answer column the model must learn to predict. train_test_split with test_size=0.25 deals 150 rows to the training fold and holds out 50 as the test fold, and random_state=0 pins the shuffle so every run — yours today, a classmate's tomorrow — deals the identical folds. With this seed, 16 of the 50 held-out applicants are defaulters. The script then prints the do-nothing baseline before any model exists: always predicting "repays" gets the other 34 right, and 34 ÷ 50 = 0.68. Write that number down; it is the score to beat.

Now the first fit, and the trap. LogisticRegression(max_iter=500) fits on the raw columns and prints accuracy 0.68, recall 0.0. Read those two numbers together: the accuracy exactly equals the do-nothing baseline, and the recall says the model flagged not one of the 16 defaulters. It has learned to predict "repays" for every single applicant. The cause is the columns' wildly different sizes: income is in the tens of thousands while debt_ratio never leaves 0 to 1, and a regularised fit across such lopsided inputs shrinks itself toward doing nothing — this model's highest predicted default probability on the test fold is about 0.37, so at the 0.5 cutoff nobody is ever flagged. Nothing crashed and nothing warned; only reading accuracy beside the baseline exposes it.

The second fit repairs the footing. make_pipeline(StandardScaler(), LogisticRegression(...)) rescales every column before fitting — StandardScaler shifts each column to mean 0 and spread 1, so a debt ratio of 0.52 and an income of $48,000 stop being a thousand-to-one mismatch — and only then fits the same model on the same folds. Accuracy jumps to 0.80 and recall to 0.438: it now catches 7 of the 16 defaulters at the price of a single false alarm (one good borrower wrongly flagged). Beating the baseline by twelve points is real signal — the four features did carry information about default; the raw fit simply could not reach it across unscaled columns.

The third fit turns the framing concept's cost table into a training instruction. class_weight="balanced" tells the fit that mistakes on the rare class cost more, in proportion to how rare it is. Recall climbs to 0.75 — 12 of the 16 defaulters caught — while accuracy falls back to 0.68, because the flagging is now aggressive enough to raise 12 false alarms. Neither fit is simply better: the scaled fit misses 9 defaulters (about $108,000 of principal at $12,000 each), while the balanced one sends 12 good borrowers into needless review. Which trade to ship is exactly the $12,000-versus-$1,400 arithmetic from the start of this lesson, and the choice is tuned on validation data — never on this test fold.

Two closing habits. First, keep the browser runtime happy: 200 rows is the right size here, and pasting a million-row extract will freeze the tab, not impress anyone. Second, before leaving, change random_state to another number and re-run: the folds re-deal and all three printed lines wobble by a few points — a small, safe demonstration of why a score quoted without its seed and split is not a result, just one shuffle's opinion.

No diagram — the evidence is the three printed accuracy and recall lines, read against the 0.68 do-nothing baseline.

Coding lab. Credit default baseline runs in the app, with checks on your output.