Reinforcement Learning · Reinforcement Learning Core
Agent, environment, state, action
In Reinforcement Learning because the agent, the floor, the cell you stand on, and the step you take are the four pieces of the loop.
Keep the same twelve tiles. Rank 1 is the top row: A1 is Start, B1 and C1 are hall tiles, D1 is Goal. Rank 2 has A2 hall, B2 Wall, C2 hall, D2 Pit. Rank 3 is four hall tiles, A3 to D3. You begin on Start. Wall cannot be stood on. Goal and Pit end the walk. This lesson names the chooser, the floor's reply, and the words state and action on those cells.
- Reinforcement Learning
- Medium level
- 6 concepts
1Name the twelve tiles
A1 is Start — that is where every walk begins. D1 is Goal — stepping there pays plus 10 and ends the walk. D2 is Pit — stepping there pays minus 10 and ends the walk. B2 is Wall — you cannot stand there; a step that would enter Wall leaves you where you were and still pays minus 1.
Every other square is Hall: B1, C1, A2, C2, and the whole of rank 3. We will keep these names for the rest of the course. When a later lesson says C1, it means the hall tile left of Goal, not a new map.
Figure. A1 is Start. D1 is Goal, plus 10. D2 is Pit, minus 10. B2 is Wall. The rest are Hall. Same floor in every later lesson.
Coding lab. Print the floor runs in the app, with checks on your output.
Which named tile ends the walk with plus 10?
- D1 Goal
- D2 Pit
- B2 Wall
Goal is D1. Pit is D2 and pays minus 10. Wall is B2 and cannot be stood on.
2The agent is the chooser
The agent is whoever picks the next step. In this course that is you — or a rule you write — standing on a named tile and choosing Up, Down, Left, or Right. From A1 the four doors are Up (off the top edge), Down (A2), Left (off the left edge), Right (B1).
The agent does not paint the floor and does not invent the payments. It only chooses. Everything else belongs to the environment, next: the named tiles, the Wall, the plus 10 and the minus 10.
Figure. The agent is the chooser. On this floor the four legal actions are Up, Down, Left, Right.
| Piece | Job |
|---|---|
| Agent | Chooses the next step |
| Environment | Moves you and pays the number |
Coding lab. The chooser picks Right runs in the app, with checks on your output.
In the reinforcement learning framework, what is the sole boundary responsibility of the agent?
- Observing the current state and selecting which action to execute next
- Simulating physics and calculating state transition probabilities
- Authoring the reward function and defining terminal goal states
- Rendering graphical visualizations of the environment grid
The agent observes state, receives reward, and selects an action. The environment governs transition physics and reward generation.
3The environment is the floor plus the rules
The environment is the named floor and the rules that go with it: what happens if you walk into Wall, off an edge, into Goal, or into Pit. You do not get to rewrite those rules mid-walk. D1 stays plus 10. B2 stays illegal to stand on.
When the agent picks Right from A1, the environment answers: you are now on B1, and the payment is minus 1. That pair — new tile, number — is the environment's whole reply. The agent chose; the floor answered.
Figure. From A1, Right. The environment answers with B1 and minus 1. The agent did not compute that pair.
The floor replies
Agent on A1 picks Right. What two things does the environment return?
- new cellB1
- reward-1
Pro tip. The environment owns the map and the payments. The agent only chose Right.
Coding lab. Read the environment's pair runs in the app, with checks on your output.
When the agent executes an action from a given state, what does the environment provide in return?
- An optimal policy table specifying all remaining steps
- The next resulting state tile and a scalar reward signal
- A full gradient vector updating all Q-values in memory
- A supervised label identifying the single best action
The environment responds to an action by emitting the next state and the immediate scalar reward.
4The state is the cell you stand on
State is a short word for 'what the agent can see that matters for the next choice.' On this floor that is which named cell you stand on. Standing on C1 is a different state from standing on A1, because Right from C1 is Goal and Right from A1 is only B1.
We will write the state as the cell name: A1, B1, C2. There is no hidden weather and no second map. If you know the cell, you know the state.
Figure. C1 highlighted: the state is 'I stand on C1'. Right from here is Goal. That is a different state from A1.
Coding lab. Print the state runs in the app, with checks on your output.
You stand on C1. Why is that a different state from A1?
- Right from C1 is Goal; Right from A1 is only B1
- C1 has a different rent label
- C1 is a neural-net layer
The cell you stand on changes which step is possible and what it pays.
5An action is one step: U, D, L, R
From any hall tile the agent picks one of four actions: Up, Down, Left, Right. Walking off an edge leaves you on the same tile and still pays minus 1 — A1 Up stays on A1. Walking into Wall does the same: A2 Right tries B2, fails, and you stay on A2 with minus 1.
Goal and Pit are terminals: once you step onto D1 or D2 the walk is over and there is no next action from there. The GIF shows one honest step from Start onto B1.
Edge and Wall stay put
From A1 take Left (off the edge). From A2 take Right (into Wall). Where do you stand, and what is paid?
- A1 --L--> (edge)A1, reward -1
- A2 --R--> (Wall B2)A2, reward -1
Pro tip. Illegal steps do not crash. They waste a turn: same cell, minus 1.
Coding lab. Bump the Wall runs in the app, with checks on your output.
If an agent on tile A1 attempts to move Left into a boundary wall, what happens in standard gridworld rules?
- The agent wraps around to the opposite side of the grid
- The episode crashes with an unhandled boundary exception
- The agent remains on tile A1 and receives the standard step penalty
- The agent is teleported directly to the terminal Goal cell
Attempting an invalid move into a boundary leaves the agent in its current state while incurring the transition cost.
6Not the AI-course agent
On this floor the agent improves because a number arrived after the last step: minus 1, plus 10, minus 10. That reward loop is the idea that is now standing. The word 'agent' is older than this course.
Artificial Intelligence already taught an agent that lives in an environment and takes actions — search, rules, logic. That agent plans a path with a known goal test. It does not improve a policy because a number arrived after the last step. Keep the AI agent in that other course.
No diagram — the idea is carried by the prose, table, code block or coding lab.
| Course | What the agent does |
|---|---|
| Artificial Intelligence | Search / logic toward a known goal test |
| Reinforcement Learning | Act, read a reward, improve a policy |
The AI lesson Agents and Environments also says 'agent'. What is different here?
- Here the agent improves from the number that arrives after it acts
- Here there is no environment
- Here the agent is a neural net by definition
The AI agent searches with a goal test. This agent learns from rewards.
Notes
- In Reinforcement Learning because the agent, the floor, the cell you stand on, and the step you take are the four pieces of the loop.
- A1 is Start — that is where every walk begins. D1 is Goal — stepping there pays plus 10 and ends the walk. D2 is Pit — stepping there pays minus 10 and ends the walk. B2 is Wall — you cannot stand there; a step that would enter Wall leaves you where you were and still pays minus 1.
- The agent is whoever picks the next step. In this course that is you — or a rule you write — standing on a named tile and choosing Up, Down, Left, or Right.
Exam traps & shortcuts
- Act, then read the number the floor paid. That number is this step, not the whole walk.
- A policy is a rule per cell. A value is how good that cell is. Neither is a neural net.
Recap
You are the agent. The named floor plus its rules is the environment. The cell you stand on is the state. The step you take is the action. The AI course's agent is search and logic, not this reward loop.
- Name the twelve tiles
- A1 is Start — that is where every walk begins. D1 is Goal — stepping there pays plus 10 and ends the walk. D2 is Pit — stepping there pays minus 10 and ends the walk. B2 is Wall — you cannot stand there; a step that would enter Wall leaves you where you were and still pays minus 1.
- The agent is the chooser
- The agent is whoever picks the next step. In this course that is you — or a rule you write — standing on a named tile and choosing Up, Down, Left, or Right. From A1 the four doors are Up (off the top edge), Down (A2), Left (off the left edge), Right (B1).
- The environment is the floor plus the rules
- The environment is the named floor and the rules that go with it: what happens if you walk into Wall, off an edge, into Goal, or into Pit. You do not get to rewrite those rules mid-walk. D1 stays plus 10. B2 stays illegal to stand on.
- The state is the cell you stand on
- State is a short word for 'what the agent can see that matters for the next choice.' On this floor that is which named cell you stand on. Standing on C1 is a different state from standing on A1, because Right from C1 is Goal and Right from A1 is only B1.
Practise Agent, environment, state, action
Reading is free and needs no account. Practice, mocks and progress live in the app.
- A 3-question practice set that ends the chapter
- 6 quick checks with worked explanations
- Timed mocks scored with the real marking scheme
- Readiness tracked per topic, kept on your device