CAS logo CAS NLP LABInteractive Tutorial Series ← Series

⌗ LECTURE 17 · LINEAR ALGEBRA — SEEING MATRICES · every transformation animated live

A matrix is motion

Forget the numbers for a second. A matrix is a motion you apply to all of space at once: it drags the grid, throws the basis vectors somewhere new, rescales every area by one factor. This page makes that motion visible and animated — drag sliders, press presets, rotate 3D space — and connects it back to the attention mechanism this series is built on.

MOTION→ PLAYGROUND→ DETERMINANT→ AB ≠ BA→ EIGENVECTORS→ 3D→ WHY HERE

THE PREVIEW — the plane, transforming on a loop

02 A MATRIX IS MOTION

Read a matrix as three facts

A 2×2 matrix \(\begin{bmatrix} a & c \\ b & d \end{bmatrix}\) tells you exactly two things: where î = (1,0) lands — the first column — and where ĵ = (0,1) lands — the second column. Those two arrows are drawn on every canvas of this page: î in blue, ĵ in red. Everything else in the plane must follow, because lines stay lines and the origin stays put — that is what “linear” means.

Columns = landing spots

The first column of the matrix is the image of î; the second is the image of ĵ. Read a matrix as two arrows and you can sketch any transformation by hand.

Linearity

Grid lines stay parallel and evenly spaced; the origin never moves. Rotations, scales, shears and reflections are all linear — translations are not (they need the extra column, §affine).

One matrix → all points

Applying the matrix to î and ĵ automatically determines where EVERY point goes: \(M\·(x\,\hat{\imath} + y\,\hat{\jmath}) = x\·M\·\hat{\imath} + y\·M\·\hat{\jmath}\). One motion, all of space, at once.

03 THE PLAYGROUND

Transform the plane with your hands

Four sliders — the four entries of the matrix. Drag them and the grid morphs in real time: the blue and red arrows are î and ĵ landing, the shaded parallelogram is the unit square’s new shape (its area = the determinant, shown live), the dashed lines are the eigendirections, and the white dot is a point you can drag to watch \(x \mapsto M\·x\). Press a preset:

THE MATRIX M (columns = î, ĵ)

det = 1.00signed area of the shaded parallelogram
—eigenvalues of the current matrix

DRAG THE WHITE DOT — its image M·x follows in gold

04 THE DETERMINANT

One number, the area factor

Every linear transformation scales all areas by the same factor — the determinant. Slide it from 1 to −1 through 0 and watch the unit square (and the whole plane) tell the story:

DETERMINANT (d of î=(1,0), ĵ=(1,d))1.00

05 COMPOSITION

Rotate-then-shear ≠ shear-then-rotate

Doing one transformation and then another is multiplying their matrices — and the ORDER matters. Two planes, two orders, same two motions:

left: rotate · right: shear — then both orders composed

S · R — shear AFTER rotate

R · S — rotate AFTER shear

06 EIGENVECTORS

The lines that refuse to turn

Most vectors get knocked off their own line by a transformation. A few don’t — they only get stretched or shrunk. Those are eigenvectors, and the stretch factor is the eigenvalue λ. Drag the matrix until the dashed lines appear:

TUNE THE MATRIX — watch the dashed eigendirections

—computed live: λ = (a+d)/2 ± √((a−d)²/4 + bc)

Why ML cares: PCA finds the eigenvectors of a data covariance matrix — the directions where the data varies most. Google’s original PageRank was an eigenvector. And in this series: attention’s value matrices transform embeddings — eigenvectors show which directions a layer preserves.

07 INTO 3D

The rotation matrix, holding a cube

In 3D a transformation matrix is 3×3 — nine numbers. Drag the three rotation sliders and read the composed matrix R = Rₓ·Rᵧ·Rᴢ that tilts the cube:

THE COMPOSED 3×3 MATRIX

08 WHY THIS MATTERS HERE

Attention runs on exactly this

Nothing in this lecture was a detour — it is the engine room of every page that came before:

Attention = weighted sums

\( \text{softmax}(QK^\top/\sqrt{d_k})\·V \): each output word is a linear combination of value vectors — the blend you animated in Step 4, with data-dependent weights.

Q, K, V are matrices

The queries, keys and values are \(X\·W^Q\), \(X\·W^K\), \(X\·W^V\) — matrix transformations of the embedding matrix X, exactly like §Step 5’s batched view.

A network is a chain of motions

Every Linear layer is \(x \mapsto W\·x + b\) — a transformation of space. Training adjusts the matrices; inference applies them. The grid you dragged today is what every embedding lives inside.

Where to go deeper

3Blue1Brown’s Essence of Linear Algebra for the animation style, Immersive Linear Algebra for the interactive-book format, and the attention deep dives of this series for the attention connection.

09 CODE LAB

The playground, runnable

NumPy morphs a grid from identity to M frame by frame; PyTorch shows that every Linear layer is one of these matrices — and composes them exactly like §05.

MATRIX MOTION · TWO WAYS

10 MASTERY CHECK

Eight questions — on seeing matrices.

YOUR SCORE

0 / 8