Eigenvalues, eigenvectors, diagonalization

Eigenvalues and eigenvectors are how we see the geometry of a matrix. A matrix that scales rather than rotates a particular direction is, in some real sense, simple along that direction. Eigendecomposition finds those directions.

Definition

A v = λ v,        v ≠ 0

v is an eigenvector and λ the corresponding eigenvalue. The defining property is that A acts on v by simple scalar multiplication — no rotation, just stretching by factor λ.

Finding eigenvalues

Rewrite Av = λv as (A - λI)v = 0. For a non-trivial v to exist, A - λI must be singular: det(A - λI) = 0. This is the characteristic polynomial, a degree-n polynomial in λ. Its n roots (with multiplicity, allowing complex roots) are the eigenvalues.

Diagonalisation

If A has n linearly independent eigenvectors, stack them as columns of a matrix V and put the eigenvalues on the diagonal of Λ:

A V = V Λ
A = V Λ V⁻¹
A^k = V Λ^k V⁻¹       (matrix powers become diagonal-easy)

Diagonalisation reduces matrix multiplication to scalar multiplication in the eigen-basis. Many otherwise-painful computations — matrix exponentials, time-evolution operators, repeated transition matrices — collapse to working component by component.

When diagonalisation fails

Not every matrix is diagonalisable. Defective matrices have fewer independent eigenvectors than eigenvalues. The classical example is [[0, 1], [0, 0]] — a single eigenvalue 0 of multiplicity 2 but only a one-dimensional eigenspace. For such matrices we need the Jordan form, which we won't need in this course because almost every matrix a quant cares about — covariance matrices, Hessians, kernel matrices — is symmetric, and symmetric matrices are always diagonalisable (Module 8).

Eigen-decomposition in finance

• Principal components of a return covariance matrix are eigenvectors of Σ, with eigenvalues equal to the variances along each principal direction (Module 10).
• The Hessian of a quadratic utility decomposes into eigen-pairs; negative eigenvalues correspond to directions of decreasing utility (Optimization Module 2).
• Markov transition matrices in credit migration models have eigen-decompositions whose top eigenvector is the stationary distribution.
• Spectral clustering of correlation networks uses the eigenvectors of the (Laplacian of the) correlation matrix to find sector blocks.