E ExamMaster

Engineering Mathematics · Probability & Statistics

Likelihood

In Engineering Mathematics because model fitting asks which parameter choice makes the observed data look most plausible under the assumed mechanism.

Probability usually treats the model as fixed and the data as uncertain. Likelihood flips that perspective: the data are fixed, and parameter values are compared by how well they explain those observations.

  • Engineering Mathematics
  • Medium level
  • 4 concepts

1Likelihood fixes the data and compares parameters

Probability asks how likely future or unknown data are under a fixed model. Likelihood uses the same model expression after the data are observed, but now the parameter value is what changes.

That viewpoint is what makes parameter estimation possible. You hold the observations still and ask which model setting makes them look least surprising.

Figure. Likelihood fixes the observed data and scores different parameter values. The highest point is the parameter best supported by this data.

Flip the viewpoint

  1. Before dataTreat the parameter as fixed and data as uncertain.
  2. After dataTreat the observed data as fixed.
  3. CompareEvaluate the data probability expression for different parameter values.

One observation under two coin models

A coin model has parameter p=P(H). The observed data are one head. Compare likelihood at p=0.5 and p=0.8.

  • Data fixedobserved H
  • L(0.5)P(H|p=0.5)=0.5
  • L(0.8)P(H|p=0.8)=0.8
  • Comparison0.8>0.5, so the one-head data favor p=0.8

Pro tip. This does not prove the coin has p=0.8; it only ranks two parameter values for the observed data.

Observed data are T from one coin toss. Compare likelihoods at p=P(H)=0.2 and p=0.7. Which is higher?
  1. p=0.2
  2. p=0.7
  3. They tie
  4. Likelihood cannot compare parameters

For a tail, the likelihood is 1-p. Values are 0.8 and 0.3, so the data favor p=0.2.

2A Bernoulli likelihood multiplies observed trial probabilities

For independent Bernoulli trials, the likelihood of a coin-bias parameter is the product of the probabilities assigned to the observed heads and tails.

The order of independent trials does not change the product once the number of heads and tails is known, so the likelihood can be written as p^k(1-p)^{n-k}.

Figure. The same HHTH data — 3 heads in 4 flips — is scored under two biases. L(0.5) = 0.0625 and L(0.75) = 0.10546875, so the data are 1.6875 times as likely under p = 0.75.

Compare coin-bias hypotheses

  1. Count dataRecord n trials and k heads.
  2. Write likelihoodUse p^k(1-p)^{n-k}.
  3. EvaluatePlug in each candidate p and compare.

HHTH under two coin biases

The observed sequence is H, H, T, H: 3 heads in 4 flips. Compare p=0.5 and p=0.75.

  • L(0.5)0.5^3(0.5)^1=0.5^4=0.0625
  • L(0.75)0.75^3(0.25)=0.421875\cdot0.25=0.10546875
  • Likelihood ratio0.10546875/0.0625=1.6875
  • Comparisonthe data are 1.6875 times as likely under p=0.75

Pro tip. Three heads out of four favors the 0.75-heads model over the fair-coin model, but by a finite evidence ratio, not certainty.

Coding lab. Compare Bernoulli likelihoods runs in the app, with checks on your output.

For data HHTTT with p=P(H), what is the Bernoulli likelihood form?
  1. p^2(1-p)^3
  2. p^3(1-p)^2
  3. 5p(1-p)
  4. p^5

The observed sequence has 2 heads and 3 tails, so multiply two head factors and three tail factors.

3Log-likelihood turns products into sums

Likelihoods multiply many probabilities, so they can become very small. Log-likelihood keeps the same ranking while turning products into sums that are easier to compute and optimize.

Machine-learning loss functions often hide this move. Negative log-likelihood means high-likelihood parameter choices become low-loss parameter choices.

Figure. Log turns the Bernoulli product at p = 0.75 into a sum: 3 ln 0.75 + ln 0.25 = −2.2494. Negative log-likelihood is 2.2494, the number a minimizer can chase.

Convert to log score

  1. ProductStart with the likelihood product.
  2. LogTake logs so multiplication becomes addition.
  3. Negate for lossUse negative log-likelihood when an optimizer minimizes.

Log-likelihood for HHTH at p=0.75

Use natural logs and the same data: 3 heads, 1 tail.

  • LikelihoodL(0.75)=0.75^3\cdot0.25
  • Log-likelihood3\ln(0.75)+\ln(0.25)
  • Numeric value3(-0.2877)+(-1.3863)=-2.2494
  • Negative log-likelihood2.2494

Pro tip. The optimizer can minimize 2.2494 instead of maximizing the tiny product; the preferred parameter ranking is unchanged.

Coding lab. Turn a likelihood product into a log sum runs in the app, with checks on your output.

Why use log-likelihood for many independent observations?
  1. It turns products into sums while preserving ranking
  2. It makes every likelihood sum to 1
  3. It removes the parameter
  4. It changes the best parameter on purpose

Log is monotone, so rankings are preserved; products of probabilities become easier additive scores.

4Maximum likelihood picks the parameter with the highest data score

Maximum likelihood estimation chooses the parameter value that gives the observed data the largest likelihood under the assumed model family.

For Bernoulli data, the maximum-likelihood estimate is the observed success fraction. That answer is simple, but it depends on accepting the independent equal-probability Bernoulli story.

Figure. The maximum likelihood estimate is the parameter at the top of the score curve, not the height itself. The argmax is the x-location.

Estimate Bernoulli p

  1. Count successesFind k successful outcomes.
  2. Count trialsFind n total outcomes.
  3. EstimateUse \hat p=k/n for the Bernoulli maximum-likelihood estimate.

MLE for four coin flips

The observed data are H, H, T, H. Compute the Bernoulli maximum-likelihood estimate.

  • Successesk=3 heads
  • Trialsn=4 flips
  • Estimate\hat p=k/n=3/4=0.75
  • Check against candidatesp=0.75 beat p=0.5 in the likelihood comparison

Pro tip. The estimate 0.75 is the best-fitting Bernoulli parameter for this tiny dataset, not a guarantee about the next four flips.

Coding lab. Find the Bernoulli MLE by a grid check runs in the app, with checks on your output.

A Bernoulli sample has 7 successes in 10 trials. What is the MLE of p?
  1. 0.7
  2. 0.3
  3. 7
  4. 10/7

For Bernoulli data, maximum likelihood picks the observed success fraction: 7/10=0.7.

Notes

  • Likelihood is the model's score for a parameter after the data are observed; it is not a probability distribution over parameters by itself.
  • Two parameter values can be ranked by likelihood even before you normalize anything.
  • Log-likelihood is often used because products of many probabilities become sums of logs.

Formulas

  • L(theta) = P(data | theta) for discrete models, or the corresponding density value for continuous models.
  • For Bernoulli data with k successes in n trials, L(p)=p^k(1-p)^(n-k).

Exam traps & shortcuts

  • Ask what is being treated as fixed before deciding whether a symbol is a probability or a likelihood.
  • Likelihood values across parameter choices are comparable scores; they do not need to sum to 1 over parameters.

Recap

Likelihood is the fitting score that compares parameters after data arrive.

Viewpoint
Probability varies data under a fixed parameter; likelihood varies the parameter for fixed data.
Coin model
For Bernoulli data, likelihood multiplies the probabilities assigned to observed heads and tails.
Logs
Log-likelihood preserves ranking while replacing products with sums.
MLE
Maximum likelihood chooses the parameter with the largest observed-data score.

Practise Likelihood

Reading is free and needs no account. Practice, mocks and progress live in the app.

  • A 5-question practice set that ends the chapter
  • 4 quick checks with worked explanations
  • Timed mocks scored with the real marking scheme
  • Readiness tracked per topic, kept on your device
Continue with Google — freeNo card, no trial. Works offline once installed.