E ExamMaster

Engineering Mathematics · Linear Algebra

Matrices as Transformations

In Engineering Mathematics because model layers, projections, and feature mixing are matrix actions on vectors, not just rectangular tables to memorize.

A matrix can be read as a compact rule that turns one vector into another. This topic frames matrices as linear transformations so later ML ideas like feature maps and layer weights have a geometric meaning.

  • Engineering Mathematics
  • Medium level
  • 5 concepts

1A matrix is a linear rule

A stockroom clerk keeps one day sheet. Each column is a part, each row is a day, and each entry is the net number of crates that moved: plus means crates came in, minus means crates went out. So in A=\begin{bmatrix}2&-1\\3&4\end{bmatrix}, day 1 reads two crates of part 1 in and one crate of part 2 out — that is what the -1 means. The table is the data: the numbers are not decoration, they are the stored rule.

Applying that stored sheet is a transformation: one list goes in and a different list comes out. Hand it the weight card x=(5,2) — what one crate of each part weighs — and the same sheet returns Ax=(8,23), the net weight moved on each day. Linear means the map respects adding lists and scaling a list — it does not invent a new rule for each input.

The shape of the sheet is a promise about what can be totalled. Two columns take exactly two weights, so when a third part moves there is no weight for it on the card and no column to put it in, and that day cannot be totalled until the sheet itself is widened.

Animation: a two-by-two stockroom day sheet sits under the heading THE DAY SHEET IS THE DATA, with part 1 and part 2 named over the columns and day 1 and day 2 beside the rows. The cells fill with 2, minus 1, 3, and 4, and each cell carries its own direction word — teal in for a positive, coral out for the negative — beside a legend reading net crates, in equals came in, out equals went out. A gold chip x equals (5, 2) arrives labelled weight card in. An arrow then carries it to a gold chip Ax equals (8, 23) labelled net weight moved each day. The closing hold reads same sheet, every time. No products, no row sweep, and no coordinate plane.
A matrix is stored data: the day sheet writes down what moved, and the sign is part of the reading — plus is crates in, minus is crates out. Applying that same sheet is a transformation — the weight card (5, 2) goes in and the net weight moved each day, (8, 23), comes out.

Read a stored day sheet

  1. Read the sheetColumns are parts and rows are days. Plus is crates in, minus is crates out.
  2. Apply the same sheetThe weight card goes in through the stored sheet. No new rule is written per input.
  3. Name the new listOut comes the net weight moved each day. Two day rows give two totals.

The day sheet acting on the weight card

A stockroom day sheet is A=\begin{bmatrix}2&-1\\3&4\end{bmatrix}, with parts down the columns and days across the rows, and the weight card is x=(5,2). Compute the net weight moved each day, Ax.

  • Day 1, first row dot x: 2\cdot5 + (-1)\cdot210-2=8
  • Day 2, second row dot x: 3\cdot5 + 4\cdot215+8=23
  • Ax(8,23)

Pro tip. The answer has two coordinates because the sheet has two day rows.

Let A=\begin{bmatrix}1&2\\0&3\end{bmatrix} and x=(4, -1). What is Ax?
  1. (2, -3)
  2. (4, -3)
  3. (2, 3)
  4. (6, -3)

Row 1: 1\cdot4+2\cdot(-1)=2. Row 2: 0\cdot4+3\cdot(-1)=-3. So Ax=(2,-3).

2Columns show where basis directions go

The columns of a matrix are not arbitrary storage positions. Column 1 is where the first basis vector lands, column 2 is where the second basis vector lands, and so on.

Once those basis images are known, every other input follows by linear combination. This is the bridge between the geometric phrase transformation and the arithmetic phrase matrix multiply.

Animation: a coordinate plane holds a gold unit arrow e1 equals (1, 0) along the x-axis and a teal unit arrow e2 equals (0, 1) along the y-axis, with matrix A drawn to the right as columns 2, 3 and -1, 4. The gold arrow travels off the axis to (2, 3), leaving a slate dot where e1 started; a chip reads Ae1 equals (2, 3), the caption reads column 1 of A, and column 1 of the matrix lights gold. The teal arrow then travels to (-1, 4); a chip reads Ae2 equals (-1, 4), the caption reads column 2 of A, and column 2 lights teal. The board changes to a weighting ledger: x equals (5, 2), then 5 times Ae1 equals 5 times (2, 3) equals (10, 15), then 2 times Ae2 equals 2 times (-1, 4) equals (-2, 8), then sum equals (8, 23). The closing hold reads the column reading and the row dot products agree. The sum is shown only as ledger rows, never as an arrow on the plane.
Column j of A is exactly where the basis vector e_j lands, so Ae1 = (2, 3) and Ae2 = (-1, 4) pin down the whole map. Every other input is then the same weighted sum: 5Ae1 + 2Ae2 = (8, 23), the answer the row dot products give.

Build from columns

  1. Read columnsFor A=[a_1\ a_2], the vectors a_1 and a_2 are Ae_1 and Ae_2.
  2. Weight columnsAn input (x_1,x_2) asks for x_1a_1+x_2a_2.
  3. AddThe weighted column sum is the same output as row multiplication.

Recover Ax from column images

For A=\begin{bmatrix}2&-1\\3&4\end{bmatrix}, use columns to compute A(5,2).

  • Ae_1(2,3)
  • Ae_2(-1,4)
  • 5Ae_1 + 2Ae_25(2,3)+2(-1,4)
  • (10,15)+(-2,8)(8,23)

Pro tip. The column view and row-dot-product view must agree; they are two readings of the same matrix.

For A=\begin{bmatrix}1&2\\0&3\end{bmatrix}, the first column is where e_1 goes. Where does e_1=(1,0) land?
  1. (1, 0)
  2. (2, 3)
  3. (0, 3)
  4. (1, 2)

Columns of A are the images of the standard basis. The first column (1,0) is Ae_1.

3Matrix composition has an order

Applying two matrix transformations in sequence is matrix composition. The rightmost transform acts on the vector first, and the next transform acts on that output.

Order usually matters. Two transformations can use the same input and the same two matrices but produce different results when the sequence is reversed.

Animation: two reference chips read B rows: (1, 2) and (0, 1), and C rows: (2, 0) and (1, 3). Below them run two horizontal pipelines, path 1 B of Cx with stage C then stage B, and path 2 C of Bx with stage B then stage C, each starting from a gold chip x equals (1, 2) and each holding an empty junction slot and an empty result slot. On the top pipeline a gold chip carrying (1, 2) slides into stage C and the junction fills with (2, 7); a chip carrying (2, 7) then slides into stage B and the result fills with (16, 7). The lower pipeline runs the same way: (1, 2) into stage B gives the junction (5, 2), which travels into stage C and gives (10, 11). The closing hold marks the two results different in coral and reads the rightmost matrix acts first, and the junction values (2, 7) and (5, 2) differ. No coordinate plane is drawn.
Composition is a pipeline and the rightmost matrix acts first, so what the second stage receives depends on the order. Same B, same C, same x = (1, 2), but the junctions (2, 7) and (5, 2) carry the two paths to (16, 7) and (10, 11).

Read a composition

  1. Start at the vectorIn BCx, compute Cx before applying B.
  2. Keep the intermediateThe output of the first transform is the input to the second.
  3. Compare only like sequencesChanging BC to CB changes the operation unless the matrices commute.

Reverse the order and compare

Let B=\begin{bmatrix}1&2\\0&1\end{bmatrix}, C=\begin{bmatrix}2&0\\1&3\end{bmatrix}, and x=(1,2). Compare B(Cx) with C(Bx).

  • Cx=(2\cdot1+0\cdot2,\ 1\cdot1+3\cdot2)(2,7)
  • B(Cx)=B(2,7)(2+14,7)=(16,7)
  • Bx=(1+4,2)(5,2)
  • C(Bx)=C(5,2)(10,5+6)=(10,11)

Pro tip. Composition is a pipeline; reversing the pipeline changes what each stage receives.

Matrices B=\begin{bmatrix}0&1\\1&0\end{bmatrix} and C=\begin{bmatrix}2&0\\0&3\end{bmatrix}. Which claim about BC versus CB is true?
  1. BC \neq CB in general, so transform order matters
  2. BC always equals CB for 2\times2 matrices
  3. Only the first matrix in a product acts on the vector
  4. Composition ignores matrix order if sizes match

Matrix multiplication is not commutative. Applying scale-then-swap differs from swap-then-scale.

4A batch is a matrix whose rows are feature vectors

In machine learning, a batch data matrix commonly stores one example per row and one feature per column. Multiplying by a weight vector then computes one dot-product score for each example.

This is matrix multiplication doing repeated vector scoring. The matrix is not merely a table; it is a compact way to run the same linear score over many rows at once.

Animation: a table opens with the headers example, feature 1, feature 2, a gold weight row reading w, 2, -1, and three teal data rows, row 1 holding 1 and 2, row 2 holding 3 and 4, row 3 holding 5 and 1. Two further columns are captioned products and score. A gold highlight band lands on row 1; an arrow carries it right and the chips 1 times 2 equals 2 and 2 times -1 equals -2 appear, then a second arrow collapses them into a score chip reading 0. The band drops to row 2, producing 3 times 2 equals 6 and 4 times -1 equals -4 and the score 2, then to row 3, producing 5 times 2 equals 10 and 1 times -1 equals -1 and the score 9. The closing hold reads Xw equals (0, 2, 9) beside the rules rows are examples and columns are features.
A batch matrix keeps one example per row and one feature per column, so Xw is a single dot product repeated down the rows. One multiply scores the whole batch: (0, 2, 9).
Batch matrix X
examplefeature 1feature 2
row 112
row 234
row 351

Three row scores at once

Let the batch rows be (1,2), (3,4), and (5,1), and let w=(2,-1). Compute Xw.

  • Row 1 dot w: 1\cdot2 + 2\cdot(-1)2-2=0
  • Row 2 dot w: 3\cdot2 + 4\cdot(-1)6-4=2
  • Row 3 dot w: 5\cdot2 + 1\cdot(-1)10-1=9
  • Xw(0,2,9)

Pro tip. Batch multiplication is just the same dot product repeated down the rows.

Coding lab. Score a batch matrix runs in the app, with checks on your output.

Batch rows are (2, 1), (0, 3), and (4, -1) with weights w=(1, 2). What are the three scores Xw?
  1. (4, 6, 2)
  2. (3, 3, 3)
  3. (4, 6, 3)
  4. (5, 6, 2)

Scores: 2+2=4, 0+6=6, 4-2=2. So Xw=(4,6,2).

5Matrices mix features into new coordinates

A matrix can also change the number and meaning of coordinates. Two original features can be mixed into two new engineered coordinates, or many features can be compressed into fewer summary coordinates.

The important habit is to name the output coordinates. After multiplication, each new coordinate is a designed combination of the input coordinates, not a mystery number.

Animation: a header reads M rows: (1, 0.5) and (-1, 2). On the left, x equals (4, 2) sits above a gold slot x1 equals 4 and a teal slot x2 equals 2; on the right are two empty output boxes, mixed 1 carrying the recipe 1 times 4 plus 0.5 times 2 and mixed 2 carrying the recipe -1 times 4 plus 2 times 2. Four hairline arrows fan from the two inputs to the two outputs. A gold chip reading 1 times 4 equals 4 travels the first arrow with its weight named beside it, and mixed 1 shows 4; a chip reading 0.5 times 2 equals 1 arrives next and mixed 1 accumulates to 5. A chip reading -1 times 4 equals -4 then travels into mixed 2, which shows -4, and a chip reading 2 times 2 equals 4 arrives so mixed 2 resolves to 0. The closing hold reads Mx equals (5, 0), with a coral line saying 0 is a real mixed value, not an empty slot, and the rules rows define the output coordinates and columns pick which input each recipe uses.
Each row of M is the recipe for one output coordinate, and each column says which input that recipe draws on. Mixed feature 2 comes out 0 because -4 and +4 cancel, so it is a computed coordinate rather than a missing one.

Name the output

  1. Input meaningState what each input coordinate measures.
  2. Row meaningRead each matrix row as the recipe for one output coordinate.
  3. Output meaningLabel the resulting coordinate before interpreting it.

Two engineered features from two inputs

A feature mixer uses M=\begin{bmatrix}1&0.5\\-1&2\end{bmatrix} on x=(4,2). Compute the two mixed features.

  • First mixed feature: 1\cdot4 + 0.5\cdot24+1=5
  • Second mixed feature: -1\cdot4 + 2\cdot2-4+4=0
  • Mx(5,0)

Pro tip. Rows define output coordinates; columns define what input positions those recipes consume.

Mixer M=\begin{bmatrix}2&1\\0&-1\end{bmatrix} acts on x=(3, 4). What is Mx?
  1. (10, -4)
  2. (6, -4)
  3. (10, 4)
  4. (7, -1)

2\cdot3+1\cdot4=10 and 0\cdot3+(-1)\cdot4=-4, so Mx=(10,-4).

Notes

  • A matrix maps an input vector to an output vector, provided the inner dimensions match.
  • Each column of a matrix shows where one basis direction is sent by the transformation.

Formulas

  • If A is m x n and x is in R^n, then A x is in R^m.
  • For a matrix A, the j-th column is A e_j, the image of the j-th standard basis vector.
  • A batch data matrix X stores examples as rows; X w computes one dot-product score per row.

Exam traps & shortcuts

  • Read the columns first when you want intuition about what a matrix does to space.
  • Check dimensions before multiplying: the input length must match the number of matrix columns.
  • Rows are examples in most ML batches, while columns are features; do not swap those roles mentally.

Recap

Think rule first, table second.

Meaning
A matrix stores a linear transformation from one vector space to another.
Columns
Each column tells you where one basis direction goes under the transformation.
Batch
A batch matrix uses rows as feature vectors, so Xw computes one score per row.
Order
In a composition, the rightmost matrix acts first; reversing the order usually changes the output.

Practise Matrices as Transformations

Reading is free and needs no account. Practice, mocks and progress live in the app.

  • A 5-question practice set that ends the chapter
  • 5 quick checks with worked explanations
  • Timed mocks scored with the real marking scheme
  • Readiness tracked per topic, kept on your device
Continue with Google — freeNo card, no trial. Works offline once installed.