Agents and Environments

1Agents and environments

An agent is anything that can be viewed as perceiving its environment through sensors and acting on it through actuators. The definition is deliberately structural: a thermostat qualifies, a chess program qualifies, a person qualifies — what varies is the sophistication of the mapping from percept history to action, not the shape of the coupling.

The boundary between agent and environment is drawn by control, not by physics or by code: the agent is exactly what chooses, and the environment is everything whose response the agent does not choose. A chess program's clock is environment — the program does not decide how time passes — and so is the opponent, and so is the user.

What crosses the boundary gets precise names. A percept is what the environment presents in one instant; the percept sequence is the complete history; an action is what the agent emits; a reward, when the framing includes one, is the environment's running score. The agent function — from percept sequence to action — is the mathematical object; the agent program is its implementation on finite hardware.

Rewards are optional, which is why supervised learning fits the framing: in this course's labs the percept is a table row, the action is a predicted label, and the graded answer plays the score's role only when we choose to keep it.

Drawing the boundary is the first modelling act, and drawing it wrong is a real failure mode: put the user inside your agent and you will 'optimise' behaviour you do not control; forget that a downstream system obeys your model's outputs and you have missed an actuator — which means missing where the damage happens.

Figure. The loop runs clockwise from the agent: it chooses an action, the action changes the environment, the environment emits the next percept (and sometimes a reward), and the percept updates the agent's next decision. In these labs the percept is a table row and the action is a prediction.

2Perceive, decide, act

The agent definition compresses into three verbs every system must implement somehow: perceive, decide, act. The triple is a checklist for reading any AI system, not an architecture — nothing says the three parts are separate modules, and in a fitted end-to-end model they are not.

A thermostat closes the loop with a sensor, a threshold comparison and a relay. A spam filter closes it with an email's text, a fitted classifier and a folder move. A self-driving stack closes it with cameras and lidar, a planner over a world model, and steering and braking commands. The same three verbs, separated by many orders of magnitude of machinery.

In this course's browser labs the loop is at its barest: the percept is one table row, the decision is the model's computation, and the action is the emitted label. Later, in the tool-using agent pattern, the identical loop runs with a language model deciding and API calls acting.

The checklist earns its keep during debugging, because failures live in different verbs and demand different fixes: a system that never receives the signal has a perception problem no amount of decision improvement will touch; a correct decision executed against a world that has since moved is an action-timing problem; and only the middle verb is ever fixed by a better model.

The picture is the Agents and environments figure read as verbs: percept arriving on the left is perceive, the agent node is decide, the action leaving on the right is act, and the environment closes the circle.

Reading a system through the loop

1. PerceiveName what evidence actually reaches the system, at what rate and delay: a sensor reading, an email's text, one row of a table.
2. DecideName what turns evidence into a choice: a threshold, a search, a fitted model. This is the only verb model training improves.
3. ActName what re-enters the world and who consumes it: a folder move, a brake command, a label a downstream system trusts.

3PEAS: specifying the task environment

Before building an agent you specify its task environment, and the standard checklist is PEAS: Performance measure, Environment, Actuators, Sensors. The acronym is mundane; the discipline is not — most disagreements about whether a system 'works' turn out to be two people holding different unwritten PEAS specifications.

The performance measure comes first deliberately: it is the success criterion stated as a condition on the world — passengers delivered safely and legally, fraud losses plus review costs minimised — and every downstream choice is judged against it. A measure written about the agent's outputs instead of the world's state re-creates the specification failures of the rationality lesson.

Environment is the space the agent operates in, including everything it cannot control: roads, weather and other drivers; a mail stream that includes adversaries writing new spam against the filter. Actuators are everything the agent can move — and in software, an emitted label that a downstream system obeys is an actuator with real force, whether or not anyone listed it. Sensors bound what the agent can ever know: a limitation here is permanent, because no decision layer recovers information the percepts never carried.

Writing the four slots down before coding exposes the classic mismatches while they are still cheap: a measure that scores beliefs instead of outcomes, an actuator with side effects nobody listed, a sensor gap the model is silently expected to guess across.

There is no picture to draw: PEAS is four labelled slots, and the table beside this concept is the figure — two systems specified side by side, slot by slot.

4Six axes of an environment

Environments differ along a small set of axes, and each axis answers one engineering question: how much machinery does the agent need? The six classical dimensions are fully versus partially observable, deterministic versus stochastic, episodic versus sequential, static versus dynamic, discrete versus continuous, and single- versus multi-agent.

Partial observability forces memory. If the current percept does not reveal the state — poker, driving in fog — the agent must maintain an internal estimate of what it cannot see, and everything downstream consumes that estimate, errors included. Full observability, as in chess with the whole board visible, is precisely what lets an agent stay memoryless.

Stochasticity forces expectation: when the same action from the same state can land differently, plans become gambles and rationality means maximising expected outcome rather than guaranteed outcome. Sequentiality forces lookahead: when today's action changes tomorrow's choices, greedy decisions compound error, and this is the pressure that makes search and planning necessary.

Dynamic environments punish slow deciders — the world moves while you deliberate, so thinking time is itself a cost. Continuous state spaces break table-based methods outright and force function approximation. Other agents make the environment adversarial or cooperative: the spam writer adapts to the filter, which is why spam filtering never stays solved.

The axes locate every lesson ahead of you: the search lessons assume observable, deterministic, static worlds — the easiest cell — and each relaxation of those assumptions is what the later machinery, probability and learning included, exists to pay for.

Imagine each axis as a slider with an easy end and a hard end. Every real task sets six sliders, and the machinery an agent needs is roughly how far right they sit — chess sets nearly all of them easy, driving sets nearly all of them hard.

5The ladder of agent architectures

Agent programs form a ladder, each rung adding one capability the previous rung lacked — and the right rung is the cheapest one your environment's axes permit, not the highest one you can build.

A simple reflex agent maps the current percept straight to an action: if temperature is below the set-point, heat. It is correct exactly when the current percept contains everything that matters — full observability — and it thrashes or loops when it does not, because it cannot remember what it can no longer see.

A model-based reflex agent adds internal state: a running estimate of the unobserved parts of the world, updated from the percept stream using a model of how the world evolves. This is the minimum machinery partial observability permits, and the estimate's errors become everyone else's inputs.

A goal-based agent adds explicit goals and chooses actions by asking what leads to them — this is where search and planning enter the story. A utility-based agent refines goals into a utility function, so it can trade off conflicting objectives and act under uncertainty by maximising expected utility: a goal only says win; a utility says how much each outcome is worth.

A learning agent adds a critic and an improvement loop, so performance rises with experience. Any of the previous rungs can be wrapped this way — which is exactly the relationship between classical AI and machine learning, drawn as an architecture rather than argued as a slogan.

Figure. Each rung exists because an environment axis forced it: hidden state forces a world model, consequential futures force goals, conflicting objectives force utility, and a drifting world forces learning. Choose the cheapest rung your environment permits — machinery above what the axes demand is surface area for bugs.

6Tool-using agents: the production loop

The production agent pattern of the current era wraps a language model in the perceive-decide-act loop: the model receives a goal and a transcript as its percepts, decides between answering and calling a tool — a search query, a code runner, a database read — and each tool's result re-enters the transcript as the next percept. The loop runs until a stop condition, and the transcript is the agent's working state.

Nothing in the framing is new. This is a model-based agent whose internal state is a text buffer and whose actuators are APIs — and that is precisely why the classical vocabulary earns its keep here: every question you can ask of a thermostat has a sharper, higher-stakes version for this loop.

The engineering risks are loop-shaped, not model-shaped. An agent with no stop condition burns budget circling; one whose tool results are fed back unvalidated will act on a tool's error as if it were ground truth; one whose actuators have irreversible side effects — sending, deleting, paying — needs deterministic guardrails between decision and execution, which is the next lesson's subject.

Capability of the wrapped model and reliability of the loop are separate properties, and they are measured separately: the same model inside a better harness — tighter tool contracts, validated observations, explicit stop conditions — is a different and usually better agent. Agent evaluation is therefore end-to-end: task completion, cost and incident rate, not the model's benchmark scores.

When you meet an agent framework, reduce it to this diagram: what are the percepts, which decisions belong to the model and which to the harness code around it, what can each tool actually do, and what stops the loop. Frameworks differ mostly in how honestly they answer.

Figure. The tool loop is the agent loop with a text buffer for state: the model reads the transcript and either answers (stop condition met) or emits a tool call; an external system executes it; the result returns as an observation appended to the transcript, which the model reads again. Guardrails belong on the tool-call edge — the one with side effects.

Auditing an agent loop

1. PerceptsWhat enters the transcript — and is any of it unvalidated tool output being treated as ground truth?
2. DecisionsWhich choices belong to the model, and which to the harness code around it? The split is the design.
3. ActuatorsWhich tool calls have side effects, and what sits between the model's decision and their execution?
4. StopWhat ends the loop — and what bounds cost and damage if nothing does?