⌗ LECTURE 17 · LINEAR ALGEBRA — SEEING MATRICES · every transformation animated live
A matrix is motion
Forget the numbers for a second. A matrix is a motion you apply to all of space at once: it drags the grid, throws the basis vectors somewhere new, rescales every area by one factor. This page makes that motion visible and animated — drag sliders, press presets, rotate 3D space — and connects it back to the attention mechanism this series is built on.
THE PREVIEW — the plane, transforming on a loop
02 A MATRIX IS MOTION
Read a matrix as three facts
A 2×2 matrix \(\begin{bmatrix} a & c \\ b & d \end{bmatrix}\) tells you exactly two things: where î = (1,0) lands — the first column — and where ĵ = (0,1) lands — the second column. Those two arrows are drawn on every canvas of this page: î in blue, ĵ in red. Everything else in the plane must follow, because lines stay lines and the origin stays put — that is what “linear” means.
The first column of the matrix is the image of î; the second is the image of ĵ. Read a matrix as two arrows and you can sketch any transformation by hand.
Grid lines stay parallel and evenly spaced; the origin never moves. Rotations, scales, shears and reflections are all linear — translations are not (they need the extra column, §affine).
Applying the matrix to î and ĵ automatically determines where EVERY point goes: \(M\·(x\,\hat{\imath} + y\,\hat{\jmath}) = x\·M\·\hat{\imath} + y\·M\·\hat{\jmath}\). One motion, all of space, at once.
03 THE PLAYGROUND
Transform the plane with your hands
Four sliders — the four entries of the matrix. Drag them and the grid morphs in real time: the blue and red arrows are î and ĵ landing, the shaded parallelogram is the unit square’s new shape (its area = the determinant, shown live), the dashed lines are the eigendirections, and the white dot is a point you can drag to watch \(x \mapsto M\·x\). Press a preset:
THE MATRIX M (columns = î, ĵ)
DRAG THE WHITE DOT — its image M·x follows in gold
04 THE DETERMINANT
One number, the area factor
Every linear transformation scales all areas by the same factor — the determinant. Slide it from 1 to −1 through 0 and watch the unit square (and the whole plane) tell the story:
05 COMPOSITION
Rotate-then-shear ≠ shear-then-rotate
Doing one transformation and then another is multiplying their matrices — and the ORDER matters. Two planes, two orders, same two motions:
S · R — shear AFTER rotate
R · S — rotate AFTER shear
06 EIGENVECTORS
The lines that refuse to turn
Most vectors get knocked off their own line by a transformation. A few don’t — they only get stretched or shrunk. Those are eigenvectors, and the stretch factor is the eigenvalue λ. Drag the matrix until the dashed lines appear:
TUNE THE MATRIX — watch the dashed eigendirections
07 INTO 3D
The rotation matrix, holding a cube
In 3D a transformation matrix is 3×3 — nine numbers. Drag the three rotation sliders and read the composed matrix R = Rₓ·Rᵧ·Rᴢ that tilts the cube:
THE COMPOSED 3×3 MATRIX
08 WHY THIS MATTERS HERE
Attention runs on exactly this
Nothing in this lecture was a detour — it is the engine room of every page that came before:
\( \text{softmax}(QK^\top/\sqrt{d_k})\·V \): each output word is a linear combination of value vectors — the blend you animated in Step 4, with data-dependent weights.
The queries, keys and values are \(X\·W^Q\), \(X\·W^K\), \(X\·W^V\) — matrix transformations of the embedding matrix X, exactly like §Step 5’s batched view.
Every Linear layer is \(x \mapsto W\·x + b\) — a transformation of space. Training adjusts the matrices; inference applies them. The grid you dragged today is what every embedding lives inside.
3Blue1Brown’s Essence of Linear Algebra for the animation style, Immersive Linear Algebra for the interactive-book format, and the attention deep dives of this series for the attention connection.
09 CODE LAB
The playground, runnable
NumPy morphs a grid from identity to M frame by frame; PyTorch shows that every Linear layer is one of these matrices — and composes them exactly like §05.
10 MASTERY CHECK