Eigenvalues and eigenvectors
Lecture 26
Eigenvalues and eigenvectors
$$ % Colors
% Coordinate vectors and matrices
% Common sets
% Abstract vector symbols
% Norms / absolute value
% Optional: dot product spacing (looks nicer in slides)
% Operators $$
Special linear transformations
- In general, describing the effect of an arbitrary linear transformation is not an easy task.
- However, we have seen some special cases where the geometric description is clear.
- For example, consider how a diagonal matrix \(D=\begin{bmatrix} a & 0 \\ 0 & b \end{bmatrix}\) changes a shape in \(\mathbb{R}^2\).
\(2\times 2\) Diagonal matrices
- In this case, the effect of \(D\) is easy to understand because:
- \(D\vec{e}_1 = a\vec{e}_1\)
- \(D\vec{e}_2 = b\vec{e}_2\)
- Note that \(\{\vec{e}_1, \vec{e}_2\}\) is an orthonormal basis, so each direction is independent.
- Therefore:
- Stretch by a factor \(a\) in the \(\vec{e}_1\) direction
- Stretch by a factor \(b\) in the \(\vec{e}_2\) direction
\(n\times n\) diagonal matrices
- Let \(D=\operatorname{diag}(\lambda_1, \lambda_2, \dots, \lambda_n)\).
- Then the geometric effect of \(D:\mathbb{R}^n \to \mathbb{R}^n\) is:
- Stretching by a factor \(\lambda_i\) in the \(\vec{e}_i\) direction for each \(i=1,2,\dots,n\)
- Since \(\{\vec{e}_1,\dots,\vec{e}_n\}\) is an orthonormal basis, each stretching is independent.
- In this case, we have \(D\vec{e}_i = \lambda_i \vec{e}_i\)
Linear transformations
- Recall that \(A=[\vec{u}_1 \ \cdots \ \vec{u}_n]\) represents a linear transformation that sends \(\vec{e}_i\) to \(\vec{u}_i\).
- We can think of \(\{\vec{u}_1,\dots,\vec{u}_n\}\) as defining a new coordinate system.
- In this coordinate system, a “diagonal-like” behavior would mean:
- \(A\vec{u}_i = \lambda_i \vec{u}_i\) for each \(i=1,2,\dots,n\)
- That is, each \(\vec{u}_i\) is simply stretched (not rotated or mixed with others).
Eigenvalues and eigenvectors
Let \(A\) be an \(n\times n\) matrix. If a nonzero vector \(\vec{u}\in\mathbb{R}^n\) satisfies \[ A\vec{u} = \lambda \vec{u} \] for some \(\lambda\in\mathbb{R}\), then:
- \(\vec{u}\) is called an eigenvector of \(A\)
- \(\lambda\) is called an eigenvalue of \(A\)
Eigenbasis
- If \(\{\vec{u}_1,\dots,\vec{u}_n\}\) is a basis consisting of eigenvectors, it is called an eigenbasis.
- In that case, \(A\) behaves like a diagonal matrix in this new coordinate system.
- Question: Can we always find an eigenbasis for any matrix \(A\)?
- Answer: Not always. We will see examples later.
Visualization
- The standard basis is not always the best frame for understanding a linear transformation.
- This visualization illustrates the difference between the standard basis and an eigenbasis.
Finding eigenvalues for 2x2
- Let \(A=\begin{bmatrix} a & b \\ c & d \end{bmatrix}\).
- Suppose \(\vec{u}=\langle x,y \rangle\neq \vec{0}\) is an eigenvector.
- Then \[ A\vec{u} = \lambda \vec{u} = (\lambda I_2) \vec{u} \]
- This is equivalent to: \[ (A-\lambda I_2)\vec{u} = \vec{0} \]
- When does this have a nonzero solution?
Characteristic equation
- A homogeneous system has a nonzero solution if and only if the matrix is singular, i.e. its determinant is zero.
- Therefore: \[ \det(A-\lambda I_2)=0 \]
- Compute: \[ \det\begin{pmatrix}a-\lambda & b \\ c & d-\lambda\end{pmatrix} = (a-\lambda)(d-\lambda)-bc = \lambda^2 - (a+d)\lambda - bc \]
- This gives a quadratic equation in \(\lambda\).
- It is called the characteristic equation.
Finding eigenvectors for 2x2
- Once an eigenvalue \(\lambda\) is found, solve: \[ (A-\lambda I_2)\vec{u} = \vec{0} \]
- Since \(\det(A-\lambda I_2)=0\), there are infinitely many solutions.
- Typically, the solution space is one-dimensional.
- Any nonzero vector in this space is an eigenvector.
Non-uniqueness of eigenvectors
- If \(\vec{u}\) is an eigenvector, then any nonzero multiple \(t\vec{u}\) is also an eigenvector.
- Indeed: \[ A(t\vec{u})=t(A\vec{u})=t\lambda\vec{u}=\lambda(t\vec{u}) \]
- Therefore, eigenvectors are not unique—only their direction matters.
Example
Let \[ A=\begin{bmatrix} 2 & 1 \\ 1 & 2 \end{bmatrix} \]
Step 1: Find eigenvalues
\[ \det(A-\lambda I_2)=\begin{vmatrix}2-\lambda & 1 \\ 1 & 2-\lambda\end{vmatrix} =(2-\lambda)^2-1 =\lambda^2-4\lambda+3 \] \[ =(\lambda-1)(\lambda-3)=0 \] So \(\lambda_1=1\), \(\lambda_2=3\)Step 2: Find eigenvectors
For \(\lambda=1\): \[ (A-I)=\begin{bmatrix} 1 & 1 \\ 1 & 1 \end{bmatrix} \] Solve \(x+y=0 \Rightarrow \vec{u}_1=\langle 1,-1 \rangle\)
For \(\lambda=3\): \[ (A-3I)=\begin{bmatrix} -1 & 1 \\ 1 & -1 \end{bmatrix} \] Solve \(-x+y=0 \Rightarrow \vec{u}_2=\langle 1,1 \rangle\)
Step 3: Geometric interpretation
- Along direction \(\langle 1,1 \rangle\): stretch by factor 3
- Along direction \(\langle 1,-1 \rangle\): stretch by factor 1 (no change)
- Along direction \(\langle 1,1 \rangle\): stretch by factor 3
So \(A\) stretches space along two special directions (its eigenvectors).
Example: No real eigenvalues
Let \[ A=\begin{bmatrix} 0 & -1 \\ 1 & 0 \end{bmatrix} \]
Step 1: Find eigenvalues
\[ \det(A-\lambda I)=\begin{vmatrix}-\lambda & -1 \\ 1 & -\lambda\end{vmatrix} =\lambda^2+1=0 \]Solutions: \[ \lambda=\pm i \]
These are not real numbers, so there are no real eigenvalues.
Conclusion:
- There are no real eigenvectors in \(\mathbb{R}^2\)
- This transformation represents a rotation by \(90^\circ\)
- No direction is preserved (everything is rotated)
Example: Repeated eigenvalue with eigenbasis
Let \[ A=\begin{bmatrix} 2 & 0 \\ 0 & 2 \end{bmatrix}=2I \]
Step 1: Find eigenvalues
\[ \det(A-\lambda I)=(2-\lambda)^2=0 \]So the only eigenvalue is \(\lambda=2\) (with multiplicity 2)
Step 2: Find eigenvectors
\[ (A-2I)=0 \]- Every nonzero vector satisfies \(A\vec{u}=2\vec{u}\)
Conclusion:
- Every direction is an eigenvector
- We can choose infinitely many eigenbases (e.g., \(\{\vec{e}_1,\vec{e}_2\}\))
- Geometrically: uniform scaling by factor 2 in all directions
Example: Repeated eigenvalue with no eigenbasis
Let \[ A=\begin{bmatrix} 1 & 1 \\ 0 & 1 \end{bmatrix} \]
Step 1: Find eigenvalues
\[ \det(A-\lambda I)=\begin{vmatrix}1-\lambda & 1 \\ 0 & 1-\lambda\end{vmatrix} =(1-\lambda)^2=0 \]So \(\lambda=1\) (with multiplicity 2)
Step 2: Find eigenvectors
\[ (A-I)=\begin{bmatrix} 0 & 1 \\ 0 & 0 \end{bmatrix} \] Solve: \[ y=0 \] So eigenvectors are: \[ \vec{u}=\langle x,0 \rangle \]Conclusion:
- Only one independent eigenvector
- Cannot form a basis of \(\mathbb{R}^2\)
- No eigenbasis exists
Geometric interpretation:
- Horizontal direction is preserved
- Other vectors are “sheared” (shifted sideways)
- This is a shear transformation, not pure stretching
Summary
- Eigenvalues and eigenvectors provide important geometric insight into a linear transformation
- If a matrix \(A\) admits an eigenbasis (a basis consisting of eigenvectors), its action can be understood as independent scaling along those directions
- Eigenvalues are obtained from the characteristic equation, and eigenvectors are found by solving \((A-\lambda I)\vec{u}=\vec{0}\)
- Not every matrix admits an eigenbasis, as illustrated by previous examples
- Next class: We will study conditions under which a matrix admits an eigenbasis