AWS Cloud Architect & Developer · AWS Cloud Architecture
Load Balancing and Auto Scaling
Elastic Load Balancing and Auto Scaling groups for resilient and scalable architectures.
Seven concepts on Elastic Load Balancing and Auto Scaling — the SAA-C03 traps are almost always classification (ALB vs NLB, which scaling policy, EC2 vs ELB health checks) plus one timing question on cooldown and grace periods.
- AWS Cloud Architect & Developer
- Medium level
- 7 concepts
- 5 practice questions
1Load balancer types
AWS offers three Elastic Load Balancer families, and the exam expects you to match workload shape to OSI layer. An Application Load Balancer (ALB) terminates HTTP and HTTPS at Layer 7 and routes on host header, path, query string, and HTTP headers — the choice for microservices and WebSocket workloads. A Network Load Balancer (NLB) handles TCP and UDP at Layer 4 with ultra-low latency, preserves the client source IP, and can expose static Elastic IPs. A Gateway Load Balancer (GWLB) sits in front of virtual appliances such as firewalls and IDS at Layer 3/4, using GENEVE encapsulation so traffic flows through inspection before reaching targets.
Figure. ALB routes HTTP/HTTPS at L7; NLB is L4 TCP/UDP; Gateway Load Balancer fronts appliances.
How each type fits
- Layer 7 content routingALB inspects HTTP and can send /api to one target group and /static to another.
- Layer 4 performanceNLB forwards TCP/UDP without parsing HTTP, scaling to millions of requests per second.
- Inline appliancesGWLB transparently chains traffic through third-party virtual appliances in the VPC.
| Type | Layer | Best for |
|---|---|---|
| ALB | 7 (HTTP/HTTPS) | Path/host routing, WebSocket, HTTP/2, cookie sticky sessions |
| NLB | 4 (TCP/UDP) | Ultra-low latency, static IP, millions of RPS, source IP preservation |
| GWLB | 3/4 (GENEVE) | Firewall, IDS/IPS, and other inline virtual appliances |
A REST API needs path-based routing (/users vs /orders) over HTTPS. Which load balancer type fits?
- Application Load Balancer
- Network Load Balancer
- Gateway Load Balancer
Path-based routing requires Layer 7 HTTP inspection, which only ALB provides. NLB forwards TCP without reading paths; GWLB chains traffic through appliances rather than routing by URL.
2Choosing ALB or NLB
Most exam scenarios reduce to a binary pick between ALB and NLB. Choose ALB when the protocol is HTTP or HTTPS and you need content-based routing, WebSocket support, or cookie-based sticky sessions. Choose NLB when the workload is raw TCP or UDP, needs the lowest possible latency, requires static Elastic IPs for firewall allow-listing, or must see the real client source IP at Layer 4 — ALB always replaces the TCP source with its own IP and exposes the client only in X-Forwarded-For on HTTP/HTTPS. Rule out ALB first when the stem mentions gaming backends, custom TCP protocols, or static IP allow-lists.
Figure. Clients connect to a stable Elastic IP on the NLB, which forwards raw TCP to game servers while preserving each client's source address — ALB never preserves TCP source IP that way; HTTP/HTTPS clients recover the address only from X-Forwarded-For.
Decision flow
- Is it HTTP/HTTPS?Yes → ALB unless you only need TCP passthrough on a known port.
- Need static IP or TCP source IP?Yes → NLB with Elastic IPs; ALB hostnames change and never preserve TCP source IP (HTTP/HTTPS uses X-Forwarded-For).
- Latency-critical TCP/UDP?Yes → NLB; Layer 7 parsing adds overhead ALB cannot avoid.
| Requirement | Why NLB | Why not ALB |
|---|---|---|
| Static IP for firewall rules | Elastic IPs attach to NLB | ALB DNS name resolves to changing IPs |
| Preserve client source IP | NLB preserves client source IP at Layer 4 | ALB replaces the TCP source with its own IP; the client appears in X-Forwarded-For |
| Custom TCP game protocol | NLB is Layer 4 — no HTTP parsing | ALB expects HTTP/HTTPS on the listener |
A multiplayer game backend needs the lowest latency, TCP handling, and a static IP clients can allow-list. Which load balancer?
- Network Load Balancer
- Application Load Balancer
- Classic Load Balancer
TCP at Layer 4 with static Elastic IPs and ultra-low latency is the NLB sweet spot. ALB is Layer 7 HTTP/HTTPS and does not offer static IPs on the load balancer itself. Classic Load Balancer is legacy and not the best answer for new designs.
3Auto Scaling group basics
An Auto Scaling group (ASG) keeps a fleet of EC2 instances between a minimum and maximum size, continuously matching a desired capacity you set or a scaling policy computes. When an instance is unhealthy or terminated, the ASG launches a replacement to hold the desired count. A launch template (or legacy launch configuration) defines the AMI, instance type, security groups, and user data every new instance receives. Spread the ASG across at least two Availability Zones so a single AZ failure never removes the entire tier — the load balancer's target group registers instances from every AZ the ASG covers.
Figure. The load balancer sends traffic straight to instances. The ASG region only marks the control plane that keeps min/desired/max across two Availability Zones and replaces unhealthy members — it is not a proxy hop.
Building a resilient tier
- Launch templateDefine AMI, instance type, IAM role, security groups, and bootstrap script once.
- Create the ASGSet min, max, and desired capacity; attach subnets in multiple AZs.
- Register with ELBAttach the ASG to an ALB or NLB target group so new instances receive traffic automatically.
- Replace unhealthyWhen health checks fail (EC2 or ELB type), terminate and launch a fresh instance.
An ASG runs web servers in a single Availability Zone behind an ALB. What is the main resilience risk?
- An AZ outage removes every instance and the ASG cannot replace them elsewhere
- The ALB cannot health-check instances in one AZ
- Auto Scaling does not work with Application Load Balancers
A single-AZ ASG has no capacity in other zones to absorb an AZ failure. Multi-AZ subnets let the ASG launch replacements elsewhere while the ALB routes only to healthy targets. ALB health checks and ASG attachment work normally in one AZ — the gap is geographic spread, not ELB compatibility.
4Scaling policies
Scaling policies tell an ASG when to add or remove instances. Target tracking is the simplest correct answer on the exam: pick a metric such as average CPU and a target such as 50%, and AWS adjusts capacity to hold the metric near that target automatically. Step scaling adds or removes capacity in graduated steps when a CloudWatch alarm breaches thresholds — useful when you want different reactions at 60%, 70%, and 80% CPU. Scheduled scaling changes capacity at fixed times for predictable patterns such as business-hours traffic spikes, and beats reactive scaling on latency when the load pattern is known in advance. Simple scaling — one fixed adjustment per alarm, then a cooldown — is legacy; it still appears when stems ask about the 300-second cooldown, but target tracking or step scaling is the preferred design answer.
Figure. Target tracking holds a metric near a setpoint; step/simple react to alarm bands; schedules fire on calendar.
Three policy shapes
- Target trackingSet a target metric value; ASG adds or removes instances to stay near it.
- Step scalingDefine alarm thresholds and how many instances to add or remove at each step.
- Scheduled scalingRaise or lower min/desired/max at cron times for known daily or weekly peaks.
| Policy | Trigger | Typical exam signal |
|---|---|---|
| Target tracking | Hold a metric at a target value | Keep average CPU around 50% |
| Step scaling | CloudWatch alarm breaches a threshold band | Add 2 instances above 70% CPU, remove 1 below 30% |
| Scheduled scaling | Fixed schedule (cron) | Scale up at 8 am weekdays, down at 6 pm |
| Simple scaling | One adjustment, then cooldown | Legacy; cooldown applies here — prefer target tracking |
A web app must automatically keep average CPU near 50% without defining step thresholds. Which scaling policy?
- Target tracking
- Step scaling
- Scheduled scaling
Target tracking holds a metric at a specified target value — exactly the keep CPU around 50% requirement. Step scaling needs explicit threshold bands and step sizes. Scheduled scaling reacts to the clock, not live CPU.
5ASG health checks: EC2 vs ELB
An Auto Scaling group decides whether to keep or replace an instance from its health check type. With the default EC2 type, the ASG trusts Amazon EC2 status checks — both StatusCheckFailed_System (underlying host or network path) and StatusCheckFailed_Instance (instance reachability / guest networking). Those checks can still pass while the application returns HTTP 503. The load balancer may already have marked the same instance unhealthy, yet the ASG keeps it in service. Switch the ASG health check type to ELB and register the group with the load balancer's target group: now the ASG replaces instances that fail the same application-level probe the ELB uses.
Figure. The same instance can pass EC2 status checks while failing the ALB's application probe. With ASG health check type EC2, traffic stops reaching the instance through the ELB but the ASG keeps it; switching to ELB ties replacement to the target group's verdict.
Two probes, one instance
- EC2 status checkSystem status covers the host path; instance status covers reachability of the guest. Neither probe asks whether the HTTP app is healthy.
- ELB target checkThe load balancer probes the app (HTTP/TCP) and marks the target unhealthy when responses fail.
- ASG with EC2 typeOnly impaired EC2 status checks trigger replacement; an app-level 503 leaves the instance in the group.
- ASG with ELB typeThe ASG watches the attached target group and terminates instances the ELB marks unhealthy.
| Health check type | What is tested | ASG replaces the instance when |
|---|---|---|
| EC2 (default) | EC2 status checks (system + instance reachability) | System or instance status is impaired, or the instance is stopped/terminated |
| ELB | Target group health (HTTP, HTTPS, TCP, or custom) | The attached load balancer marks the instance unhealthy in its target group |
A web tier sits behind an ALB. Targets fail the ALB HTTP health check with 503, but EC2 status checks pass and the ASG is not replacing instances. What change fixes this?
- Switch the ASG health check type from EC2 to ELB
- Increase the ASG desired capacity
- Disable ALB health checks on the target group
- Enable sticky sessions on the ALB
EC2 status checks say the instance is reachable; they do not know the app returns 503. The ASG with EC2 health check type keeps such instances until you switch to ELB and attach the target group, so replacement follows the same app probe the ALB uses. Adding capacity spreads traffic across more broken instances; disabling health checks routes to dead apps; sticky sessions bind clients to one target but do not replace it.
6Sticky sessions on ALB
When an application stores session state locally on one server — shopping carts in memory, WebSocket rooms, or server-side session files — a load balancer must send the same client back to the same target on every request. An Application Load Balancer supports sticky sessions (session affinity) by inserting a cookie: either the ALB generates an AWSALB cookie or the application provides its own cookie name for the load balancer to track. Network Load Balancers offer a different affinity mechanism — source IP stickiness at the target group — not cookie-based HTTP sessions. Cookie stickiness still trades horizontal scalability for per-client binding: if that target dies, the client's session is lost unless the app externalizes state to ElastiCache or a database.
Figure. Stickiness pins a client cookie to one target — useful for session memory, harmful for even load.
How affinity works
- First requestALB picks a healthy target and, if stickiness is enabled, sets an AWSALB cookie on the response.
- Later requestsThe client sends the cookie back; ALB routes to the same target until the stickiness duration expires.
- Target failureIf the bound target becomes unhealthy, ALB picks a new target and issues a fresh cookie — local session state is lost unless externalized.
A stateful web app keeps user sessions in local server memory. Traffic is behind an ALB. What ALB feature keeps a user on the same instance?
- Enable sticky sessions (session affinity) on the target group
- Switch the ASG health check type to EC2
- Enable cross-zone load balancing on the NLB
- Add a scheduled scaling policy
Sticky sessions bind a client to one target via an ALB cookie — the standard fix for in-memory session state behind an ALB. EC2 health checks ignore application sessions; cross-zone on an NLB and scheduled scaling do not bind an HTTP client to one app instance.
7Cooldown and health check grace
Two default 300-second timers appear on ASG exam stems, but they answer different questions. The health check grace period gives a newly launched instance 300 seconds after launch before the ASG evaluates its health — time for boot scripts and application startup without premature termination. The default cooldown (300 seconds) applies to simple scaling policies: after a simple scaling action completes, the ASG ignores further alarm-driven scale-in or scale-out until cooldown expires. Target tracking and step scaling do not wait on that cooldown — they use instance warmup instead. Cross-zone load balancing distributes requests evenly across AZs: enabled by default and free on ALB; on NLB it is optional and incurs inter-AZ data charges when enabled.
Figure. Both timers default to 300 seconds but answer different questions. Grace protects a booting instance under every policy; cooldown only throttles simple scaling while metrics settle.
Two timers, different jobs
- Grace period (300 s default)After launch, ASG ignores failed health checks until grace expires — boot time for the app.
- Cooldown (300 s default)After a simple scaling action completes, ASG waits before another simple-scaling scale-in or scale-out. Target tracking and step scaling ignore this timer and use instance warmup.
- Cross-zone on ALB vs NLBALB cross-zone is on by default at no extra charge; NLB cross-zone is off by default and costs for inter-AZ traffic when enabled.
| Timer | Starts when | Purpose |
|---|---|---|
| Health check grace period | Instance enters InService | Allow boot and app startup before health evaluation |
| Cooldown period | Simple scaling action completes | Block further simple-scaling events while metrics settle — not used by target tracking or step scaling |
When cooldown blocks a second scale-out
An ASG using a simple scaling policy with the default 300 s cooldown scales out at t = 0 s. CloudWatch alarms for high CPU fire at t = 120 s and again at t = 280 s while CPU stays high. When can the ASG scale out again under that simple scaling policy?
- Simple scale-out completes at t = 0, cooldown = 300 sblocked until t = 300 s
- Alarm at t = 120 s (120 < 300)ignored — still in cooldown
- Alarm at t = 280 s (280 < 300)still ignored
- Earliest next simple scale-out at t = 300 scooldown expired
Pro tip. Grace period protects new instances under every policy type. Cooldown protects only simple-scaling fleets from thrashing — target tracking and step scaling use instance warmup instead. Exam stems that mention boot time point to grace; stems that mention simple scaling and repeated alarms point to cooldown.
A new EC2 instance takes 4 minutes to boot its application. The ASG terminates it after 90 seconds because health checks fail during startup. What setting helps?
- Increase the health check grace period
- Increase the cooldown period
- Disable cross-zone load balancing
- Switch to step scaling
The grace period delays health evaluation after launch so slow-booting apps can pass checks. Cooldown governs time between simple scaling actions, not startup tolerance. Cross-zone and step scaling do not extend the boot window.
Notes
- Load Balancer Types: Application Load Balancer (ALB) operates at Layer 7 for HTTP/HTTPS with content-based routing; Network Load Balancer (NLB) operates at Layer 4 for ultra-low latency and static IPs; Gateway Load Balancer fronts virtual appliances.
- Auto Scaling Groups: An ASG maintains a desired count of EC2 instances between min and max sizes, replacing unhealthy instances and scaling on policies or schedules.
- Scaling Policies: Target tracking keeps a metric (e.g., 50% CPU) at target; step scaling adds/removes capacity in graduated steps; scheduled scaling handles predictable patterns.
- Health Checks: ELB routes only to healthy targets; ASG can use ELB health checks so instances failing app-level checks are replaced, not just EC2 status checks.
- Sticky Sessions: ALB supports session affinity via cookies so a client stays bound to one target when the app is stateful.
Formulas
- ALB routing: content-based rules on host header, path, query string, and HTTP headers; supports WebSocket and HTTP/2.
- NLB scale: handles millions of requests per second with static/Elastic IP support and preserves the source IP.
- ASG cooldown: default 300 seconds to let metrics stabilize before another scaling action.
- Health check grace period: default 300 seconds after launch before ASG evaluates instance health.
- Cross-zone load balancing: enabled by default and free on ALB; optional (and charged) on NLB.
Exam traps & shortcuts
- Layer-7 path/host routing to microservices => ALB; extreme performance, static IP, or TCP/UDP => NLB.
- If instances are unhealthy at the app layer but ASG isn't replacing them, switch the ASG health check type from EC2 to ELB.
- Use target tracking scaling for the simplest correct answer to 'keep CPU around X%.'
- For predictable daily/weekly spikes (e.g., business hours), scheduled scaling beats reactive scaling on latency.
Reference tables
| Topic | Default or rule | Exam trap |
|---|---|---|
| ALB layer | Layer 7 HTTP/HTTPS | Not for raw TCP gaming protocols |
| NLB layer | Layer 4 TCP/UDP, static EIP | Source IP preserved; ALB is not |
| ASG health check | EC2 by default | Switch to ELB when app checks fail but EC2 status passes |
| Target tracking | Hold metric at target | Simplest answer for keep CPU at X% |
| Grace period | 300 s after launch | Boot time before health evaluation |
| Cooldown | 300 s after simple scaling action | Does not gate target tracking or step scaling — those use instance warmup |
| Cross-zone LB | On by default on ALB (free) | Optional on NLB (inter-AZ charge) |
| Sticky sessions | ALB cookie affinity | NLB has source-IP affinity — not cookie stickiness |
Recap
Read this before a practice test on load balancing and Auto Scaling.
- LB types
- ALB for HTTP routing, NLB for TCP/UDP with static IP and source IP, GWLB for inline appliances.
- ALB vs NLB
- HTTP microservices to ALB; gaming TCP, static IP, or source IP to NLB.
- ASG
- Min/desired/max across multiple AZs; launch template defines every new instance.
- Scaling
- Target tracking for keep metric at X%; scheduled for clock-time peaks; step for graduated alarm bands; simple scaling is the legacy cooldown case.
- Health checks
- EC2 type uses system + instance status checks — not application HTTP health; ELB type replaces instances the load balancer marks unhealthy.
- Stickiness
- ALB cookie affinity binds HTTP clients to one target; NLB uses source-IP affinity instead. Externalize session state for resilience.
- Timers
- 300 s grace for boot; 300 s cooldown after simple scaling only — target tracking and step use instance warmup. Cross-zone free on ALB, optional on NLB.
Practise Load Balancing and Auto Scaling
Reading is free and needs no account. Practice, mocks and progress live in the app.
- 5 exam-style questions on this topic, with explanations
- A 6-question practice set that ends the chapter
- Timed mocks scored with the real marking scheme
- Readiness tracked per topic, kept on your device