E ExamMaster

Engineering Mathematics · Optimization

Local and Global Minima

In Engineering Mathematics because optimization quality depends on whether a low point is merely nearby-best or truly best overall.

Not every low point gives the same guarantee. This topic separates a point that is best nearby from one that is best across the whole search space.

  • Engineering Mathematics
  • Medium level
  • 4 concepts

1A local minimum is only a neighborhood guarantee

A local minimum is lower than nearby feasible points. It can still be worse than another valley far away.

Gradient-based methods are especially sensitive to this distinction because they use local slope information rather than an exhaustive scan of the whole domain.

Figure. A local minimum is lower than nearby points, but another valley may be lower elsewhere. The plot separates neighborhood-best from overall-best.

Check locality

  1. Find stationary pointSolve or inspect where the derivative is zero.
  2. Check nearbyUse curvature or nearby values to see whether the point is a low point.
  3. Limit the claimCall it local unless all feasible points have been compared.

One valley is local but not global

For f(w)=\frac{1}{4}w^4-\frac{1}{3}w^3-w^2, the derivative is f'(w)=(w+1)w(w-2). Evaluate the two minimum candidates w=-1 and w=2.

  • f(-1)\frac{1}{4}+\frac{1}{3}-1=-\frac{5}{12}\approx-0.417
  • f(2)4-\frac{8}{3}-4=-\frac{8}{3}\approx-2.667
  • Lower of these two valleysw=2

Pro tip. Both points can be local minima, but only the lower one is global among the candidates compared here.

A local minimum means:
  1. No nearby feasible point has smaller loss
  2. It is best over the entire domain
  3. Gradient is undefined
  4. Loss is negative

Local = neighborhood claim only.

2Global means best over the full feasible set

A global minimum is not certified by being low or by having zero gradient. It is certified by being no worse than every feasible alternative.

For convex objectives the local-to-global step is often guaranteed by shape. For non-convex objectives, the same claim needs stronger evidence.

The middle critical point is a peak, while the two outer critical points are valleys with different depths.

Raise the guarantee

  1. Local testCheck derivative and nearby behavior.
  2. Compare valleysEvaluate all relevant low points or use a theorem.
  3. State scopeSay whether the guarantee is local or global.

Same derivative condition, different guarantee

For the same f, critical points are -1, 0, and 2. Use second derivative f''(w)=3w^2-2w-2 and values to classify them.

  • f''(-1)=3+2-23>0, local minimum
  • f''(0)-2<0, local maximum
  • f''(2)=12-4-26>0, local minimum
  • Values at minimaf(2)=-8/3<f(-1)=-5/12

Pro tip. Derivative zero found candidates; curvature classified them; objective values decided which local minimum is lower.

Global minimum requires:
  1. Best value among all feasible candidates, not just a neighborhood
  2. Only that L'=0
  3. A positive Hessian everywhere
  4. Stochastic gradients

Global compares the whole feasible set.

3Starting point can decide which valley is reached

On a non-convex objective, two runs of the same optimizer can end in different valleys because their starting points see different local slopes.

This is one reason ML training may use restarts, initialization rules, or objectives designed to make bad local traps less likely.

Figure. Different starting points can fall into different valleys on a nonconvex landscape. Initialization matters because descent is local.

Read the trap

  1. Start pointThe first gradient is computed where the run begins.
  2. BasinThe sequence may stay in the valley system reached from that start.
  3. RestartAnother start can explore another basin.

Why a shallow valley can still capture a run

Suppose one run ends at w=-1 and another at w=2 for the same bimodal f. Compare the final objective values.

  • Run A final valuef(-1)=-5/12\approx-0.417
  • Run B final valuef(2)=-8/3\approx-2.667
  • Value gap-5/12-(-8/3)=9/4=2.25

Pro tip. A run that reaches the shallow local minimum is not broken; it followed local information to a valley that was not the best one.

Coding lab. Two starts, two valleys runs in the app, with checks on your output.

Two runs of descent reach different valleys mainly because:
  1. Different starts can fall into different basins
  2. Loss functions forbid multiple minima
  3. Initialization cannot matter
  4. Learning rate is unused

Nonconvex landscapes make the start pick a basin.

4A zero gradient can be a saddle

A zero gradient means the first-order slope has vanished. It does not by itself prove the point is a minimum.

In more than one dimension, a saddle can curve upward in one direction and downward in another, so local tests must look beyond the gradient vector.

A saddle surface would rise along the x direction and fall along the y direction at the same central point.

Test a saddle

  1. Set gradient zeroFind the stationary point.
  2. Move two waysCheck nearby values along different directions.
  3. ClassifyIf one nearby direction is lower and another higher, it is not a minimum.

The origin is not a minimum

For g(x,y)=x^2-y^2, evaluate near (0,0) along the x and y axes.

  • g(0,0)0
  • g(0.1,0)0.01
  • g(0,0.1)-0.01
  • ClassificationNearby values are both above and below 0

Pro tip. The gradient at the origin is zero, but the point is not a minimum because moving in the y direction lowers the objective.

For g(x,y)=x^2-y^2 at (0,0), the critical point is a saddle because:
  1. Along x it looks like a min, along y like a max
  2. Gradients never vanish there
  3. It is a global min
  4. Hessians are illegal

Mixed curvature: not a local min.

Notes

  • A local minimum beats nearby points; a global minimum beats all feasible points.
  • High-dimensional objectives can also contain saddle points that are neither minima nor maxima overall.
  • A zero gradient identifies a candidate point, not automatically the best possible model.

Formulas

  • A local minimum is best in a neighborhood; a global minimum is best over the full feasible set.

Exam traps & shortcuts

  • When a method gets stuck, it may be reacting to local geometry rather than failing to follow the update rule.
  • Evaluate candidate values before calling one minimum global.

Reference tables

Use the guarantee that the evidence supports.

Minimum vocabulary
ClaimEvidence neededRisk
Stationary pointf'(w)=0Could be a peak or saddle
Local minimumNearby points are no lowerAnother valley may be lower
Global minimumNo feasible point is lowerNeeds full comparison or a theorem

Recap

A minimum claim needs a scope: nearby or everywhere.

Local
A local minimum is lower than nearby alternatives.
Global
A global minimum is lower than every feasible alternative.
Bimodal example
For f(w)=\frac{1}{4}w^4-\frac{1}{3}w^3-w^2, f(-1)=-5/12 and f(2)=-8/3, so w=2 is the lower valley.
Starting point
Local slope methods can land in different valleys from different initializations.

Practise Local and Global Minima

Reading is free and needs no account. Practice, mocks and progress live in the app.

  • A 5-question practice set that ends the chapter
  • 4 quick checks with worked explanations
  • Timed mocks scored with the real marking scheme
  • Readiness tracked per topic, kept on your device
Continue with Google — freeNo card, no trial. Works offline once installed.