E ExamMaster

Engineering Mathematics · Probability & Statistics

Expectation, Variance and Covariance

In Engineering Mathematics because mean behavior, spread, and co-movement are the first summaries used to reason about data and risk.

Once a distribution exists, the next step is summarizing it. This topic computes center, spread, and co-movement from small tables so mean behavior and uncertainty become auditable quantities.

  • Engineering Mathematics
  • Medium level
  • 4 concepts

1Expectation is a probability-weighted average

Expectation is the average value a distribution implies over repeated draws. Each possible value contributes in proportion to its probability mass.

This is why rare large values can matter. In data and risk work, the expected value is a distribution-level summary, not the most likely single observation.

Figure. Expectation sums signed weighted contributions, not just raw outcomes. The positive contributions dominate the small negative one to produce E[X]=1.60.

Compute expectation

  1. MultiplyMultiply each value by its probability.
  2. AddAdd the weighted contributions.
  3. InterpretRead the result as a long-run average under the model.
Point mass table
xP(X=x)xP(X=x)
-10.10-0.10
00.200.00
20.400.80
30.300.90

Expected score from a four-point distribution

A scoring error X has values -1, 0, 2, 3 with probabilities 0.10, 0.20, 0.40, 0.30. Compute E[X].

  • Weighted values-1\cdot0.10,\ 0\cdot0.20,\ 2\cdot0.40,\ 3\cdot0.30
  • Contributions-0.10,\ 0.00,\ 0.80,\ 0.90
  • Sum-0.10+0.00+0.80+0.90=1.60

Pro tip. The expected score is 1.60 even though 1.60 is not a possible realized value.

Coding lab. Compute expectation from a mass table runs in the app, with checks on your output.

A variable takes 0,2,5 with probabilities 0.2,0.5,0.3. What is E[X]?
  1. 2.5
  2. 7
  3. 2.33
  4. 1.0

Compute the weighted average: 0(0.2)+2(0.5)+5(0.3)=0+1+1.5=2.5.

2Variance measures squared spread around the mean

Variance summarizes how far a random variable tends to sit from its mean, using squared distance so positive and negative deviations do not cancel.

A convenient discrete computation is Var(X)=E[X^2]-(E[X])^2. The first term averages squared values; the second removes the square of the center.

Figure. Variance is the leftover after subtracting the square of the mean from the second moment. The bars make the subtraction target explicit.

Compute variance

  1. Second momentCompute E[X^2] from the same mass table.
  2. Mean squareSquare the already computed mean.
  3. SubtractUse E[X^2]-(E[X])^2.
Second moment ledger
xP(X=x)x^2P(X=x)
-10.100.10
00.200.00
20.401.60
30.302.70

Variance from the same four-point table

Using the distribution from the expectation example, E[X]=1.60. Compute Var(X).

  • E[X^2]1\cdot0.10+0\cdot0.20+4\cdot0.40+9\cdot0.30
  • E[X^2]0.10+0+1.60+2.70=4.40
  • (E[X])^21.60^2=2.56
  • Var(X)4.40-2.56=1.84

Pro tip. The variance is 1.84; it is not 4.40 and not 1.60^2.

For a variable with E[X]=3 and E[X^2]=13, what is Var(X)?
  1. 4
  2. 10
  3. 16
  4. 13

Variance is E[X^2]-(E[X])^2=13-9=4. Do not subtract the mean only once.

3Covariance tracks joint movement around two means

Covariance is about two variables together. It is positive when high values of one tend to come with high values of the other and low values tend to come together.

The sign is often more important than the raw size at first. Scale changes can stretch covariance, so use it to detect directional co-movement before comparing magnitudes across unrelated units.

Figure. The paired points trend upward: larger x values tend to arrive with larger y values. Covariance averages that centered co-movement.

Compute sample covariance

  1. MeansFind the average of each variable.
  2. DeviationsSubtract each mean row by row.
  3. ProductsAverage the row-wise deviation products.
Four paired observations
rowxy(x-2)(y-3)
1121
2230
3352
4220

Covariance of tiny feature pairs

Four observations are (1,2), (2,3), (3,5), (2,2). Use population averaging for this teaching ledger.

  • Means\bar x=(1+2+3+2)/4=2, \bar y=(2+3+5+2)/4=3
  • Deviation products1, 0, 2, 0
  • Average product(1+0+2+0)/4=0.75

Pro tip. The positive covariance says larger x tends to arrive with larger y in this tiny table.

Coding lab. Center paired observations and compute covariance runs in the app, with checks on your output.

If paired deviations are (x-μ_x)=(1,-1,2,-2) and (y-μ_y)=(3,-3,1,-1), what is the sign of covariance?
  1. Positive
  2. Negative
  3. Zero
  4. Impossible to tell

Row-wise products are 3,3,2,2, all positive, so the average product is positive.

4These summaries become model diagnostics

Means, variances, and covariances are not isolated textbook quantities. They show up when standardizing features, reading noise levels, checking feature redundancy, and building covariance matrices.

The ML habit is to connect each number to a decision. A large variance may demand scaling, and a strong covariance may warn that two features carry overlapping movement.

Figure. Standardize by centering then scaling. Value 13 minus mean 10 is 3. Variance 4 gives standard deviation 2, so the z-style value is 3/2 = 1.5.

Read a feature summary

  1. CenterUse the mean to locate the feature level.
  2. ScaleUse variance or standard deviation to judge spread.
  3. Co-movementUse covariance to spot directional overlap between features.

Standardize a feature value

A feature has mean 10 and variance 4. A new value is 13. Compute its z-style standardized value using standard deviation.

  • Standard deviation\sqrt{4}=2
  • Centered value13-10=3
  • Standardized value3/2=1.5

Pro tip. Standardization turns the raw value into 1.5 standard deviations above the feature mean.

A feature has mean 50 and standard deviation 10. What is the standardized value of a new value 65?
  1. 1.5
  2. 15
  3. 6.5
  4. 0.15

Standardize by subtracting the mean and dividing by standard deviation: (65-50)/10=1.5.

Notes

  • Expectation is the long-run average value implied by the distribution, not a guarantee for one observation.
  • Variance measures squared spread around the mean.
  • Covariance measures whether two variables tend to move above or below their means together.

Formulas

  • E[X] = sum_x x P(X = x).
  • Var(X) = E[X^2] - (E[X])^2.
  • Cov(X,Y) = E[XY] - E[X]E[Y].

Exam traps & shortcuts

  • Compute E[X^2] separately; squaring E[X] is not variance.
  • A positive covariance says the variables tend to move together; it does not say they move on the same scale.

Recap

Expectation, variance, and covariance turn distributions into auditable summaries.

Expectation
Expectation is a probability-weighted average, not a guaranteed draw.
Variance
Variance needs squared spread around the mean, often via E[X^2]-(E[X])^2.
Covariance
Covariance averages paired deviations to reveal directional co-movement.
ML use
Feature scaling and redundancy checks depend on these summaries being interpreted correctly.

Practise Expectation, Variance and Covariance

Reading is free and needs no account. Practice, mocks and progress live in the app.

  • A 5-question practice set that ends the chapter
  • 4 quick checks with worked explanations
  • Timed mocks scored with the real marking scheme
  • Readiness tracked per topic, kept on your device
Continue with Google — freeNo card, no trial. Works offline once installed.