Engineering Mathematics · Probability & Statistics
Sampling, Bias and Variance
In Engineering Mathematics because data quality and estimator stability matter before any sophisticated model choice does.
Statistics is never only about formulas on paper. The sample you collect and the repeatability of your estimator determine whether an answer is trustworthy in the first place.
- Engineering Mathematics
- Medium level
- 4 concepts
1A sample is the part of the population you actually see
A population is the full group you care about; a sample is the subset that enters the dataset. Sampling is not clerical because it decides which cases are allowed to influence every later calculation.
In engineering data work, logs, surveys, sensors, and clickstreams are often convenient samples. Convenience can be useful, but it must not be mistaken for representation.
Figure. A survey reaches 2,000 lesson finishers out of 10,000 active learners (coverage 0.20), then 200 responders (0.10 of finishers). Direct population response is 0.02. The boxes are equal size, not area-scaled.
Audit the sample
- PopulationName the group the conclusion is about.
- Sampling pathName how observations entered the dataset.
- Missing groupAsk which population members have lower or zero chance of appearing.
App survey sample coverage
A learning app has 10,000 active learners. A survey is shown only after completing a lesson; 2,000 learners finish a lesson and 200 of them respond. Compute the population coverage and finisher response rate.
- Population coverage by eligible finishers2000/10000=0.20
- Response among finishers200/2000=0.10
- Direct population response200/10000=0.02
- Sampling warning98 percent of active learners did not directly enter the survey data
Pro tip. The survey may still be useful, but it mainly represents responders among finishers, not all active learners.
A course has 800 learners, but feedback is collected from 120 who finished the final project. What is direct population response coverage?
- 120/800=0.15
- 120/680=0.176
- 680/800=0.85
- 800/120=6.67
Coverage uses the target population as denominator: 120 responses out of 800 learners.
2Bias is average error in one direction
Bias measures whether an estimator is systematically pulled away from the true value. It is defined over repeated use of the method, not over one lucky or unlucky sample.
Sampling bias is one source: if important cases are under-represented, the average estimate can miss the population target even when each calculation is performed correctly.
Figure. Bias is a systematic offset: the estimates cluster on one side of the true value rather than merely bouncing around it.
Compute bias
- Repeated estimatesImagine or measure the estimator across repeated samples.
- Average estimateCompute the mean of those estimates.
- Subtract truthBias equals average estimate minus the true value.
| sample | estimate A | estimate B |
|---|---|---|
| 1 | 8 | 10 |
| 2 | 9 | 9 |
| 3 | 7 | 10 |
| 4 | 8 | 9 |
Bias from four repeated estimates
The true value is 8. Estimator A gives 8, 9, 7, 8. Estimator B gives 10, 9, 10, 9.
- Mean of A(8+9+7+8)/4=8
- Bias of A8-8=0
- Mean of B(10+9+10+9)/4=9.5
- Bias of B9.5-8=1.5
Pro tip. Estimator B is systematically high by 1.5 even though its estimates look tightly grouped.
An estimator averages 14,15,13,14 when the true value is 12. What is its bias?
- 2
- 14
- -2
- 1
The repeated-estimate mean is 14, so bias is 14-12=2.
3Variance is how much estimates swing across samples
Estimator variance measures instability across repeated samples. An unbiased method can still be noisy if its answer changes a lot when the sample changes.
This is different from bias. Bias asks where the repeated estimates center; variance asks how widely they scatter around that center.
Figure. Variance is instability across repeated samples. The high-variance estimates swing more even when the target is the same.
Compute repeated-estimate variance
- Mean estimateFind the average of the repeated estimates.
- Squared deviationsSquare each estimate's distance from that mean.
- AverageAverage those squared deviations for a simple teaching ledger.
Variance of estimators A and B
Use the repeated estimates A = 8, 9, 7, 8 and B = 10, 9, 10, 9.
- A mean and deviations8 with deviations 0,1,-1,0
- A variance(0^2+1^2+(-1)^2+0^2)/4=0.5
- B mean and deviations9.5 with deviations 0.5,-0.5,0.5,-0.5
- B variance(0.25+0.25+0.25+0.25)/4=0.25
Pro tip. B has lower variance but higher bias; stability alone did not make it better.
Estimates 4,6,4,6 have mean 5. What is the average squared deviation?
- 1
- 2
- 0
- 5
Deviations are -1,1,-1,1; squared deviations are all 1, so the average is 1.
4Model choices often trade bias against variance
A very simple model may be stable but systematically miss structure, creating bias. A very flexible model may chase sample noise, creating high variance.
The point is not that one side is always good. The engineering task is to choose a method whose average error and sample-to-sample instability are both acceptable for the decision being made.
Figure. As flexibility rises, bias tends to fall while variance tends to rise. The total error curve marks the compromise region.
Read the tradeoff
- Simple methodCheck whether it underfits the true pattern on average.
- Flexible methodCheck whether it changes too much across samples.
- DecisionUse validation or repeated-sample evidence to balance the two errors.
| method | bias | variance | risk reading |
|---|---|---|---|
| A | 0.0 | 0.5 | centered but noisier |
| B | 1.5 | 0.25 | stable but too high |
Mean squared error from bias and variance
Using the teaching decomposition MSE = bias^2 + variance, compare estimator A with bias 0 and variance 0.5 against estimator B with bias 1.5 and variance 0.25.
- A MSE0^2+0.5=0.5
- B MSE1.5^2+0.25=2.25+0.25=2.5
- Comparison0.5<2.5, so A wins on this error summary
Pro tip. The lower-variance estimator loses because its squared bias is too costly.
Coding lab. Bias–variance sketch on a 1-D fit runs in the app, with checks on your output.
Method A has bias 0.5 and variance 0.75. Using bias^2+variance, what is its teaching MSE?
- 1.0
- 1.25
- 0.875
- 0.25
0.5^2+0.75=0.25+0.75=1.0. Bias is squared before adding variance.
Notes
- Sampling decides which part of the population enters the dataset and can therefore inject systematic distortion.
- Bias tracks systematic error: average estimate minus the true value.
- Variance tracks instability across repeated samples.
Formulas
- Bias(estimator) = E[estimator] - true value.
- Variance across repeated estimates = average squared distance from their mean.
Exam traps & shortcuts
- A precise answer can still be wrong if the sample is unrepresentative.
- Low variance is not enough; a consistently wrong estimator can be stable and biased.
Recap
Separate data-collection distortion from estimator instability.
- Sampling
- The sample is the part of the population allowed to influence the conclusion.
- Bias
- Bias is repeated-estimate center minus the true value.
- Variance
- Variance is repeated-estimate spread around that center.
- Tradeoff
- A stable estimator can still be wrong if bias dominates the error.
Practise Sampling, Bias and Variance
Reading is free and needs no account. Practice, mocks and progress live in the app.
- A 5-question practice set that ends the chapter
- 4 quick checks with worked explanations
- Timed mocks scored with the real marking scheme
- Readiness tracked per topic, kept on your device