Orthogonality

Lecture 16

Minjae Park

Auburn University
MATH 2660 - Spring 2026

February 16, 2026

Attendance

Scan the QR code or go to join.iclicker.com/MBNJ.

The dot product (also called the scalar product or inner product) is defined for two vectors \(\vec{u},\vec{v}\in\mathbb{R}^n\).
The result is a scalar (a real number).
If \[ \vec{u}=\langle u_1,\ldots,u_n \rangle, \quad \vec{v}=\langle v_1,\ldots,v_n \rangle, \] then \[ \vec{u}\!\cdot\!\vec{v} = u_1v_1+\cdots+u_nv_n. \]
The dot product measures length and angle.

\(\vec{u}\!\cdot\!\vec{v}=\vec{v}\!\cdot\!\vec{u}\) (commutativity)
\(\vec{u}\!\cdot\!(\vec{v}+\vec{w})=\vec{u}\!\cdot\!\vec{v}+\vec{u}\!\cdot\!\vec{w}\) (distributivity)
\((c\vec{u})\!\cdot\!\vec{v}=c(\vec{u}\!\cdot\!\vec{v})\) (compatibility with scalars)
\(\vec{u}\!\cdot\!\vec{u}\ge 0\), and \(\vec{u}\!\cdot\!\vec{u}=0\) iff \(\vec{u}=\vec{0}\) (positivity)
\(\left\lVert \vec{u} \right\rVert^2=\vec{u}\!\cdot\!\vec{u}\) (length from dot product)
\(\displaystyle \cos\theta=\frac{\vec{u}\!\cdot\!\vec{v}}{\left\lVert \vec{u} \right\rVert\,\left\lVert \vec{v} \right\rVert}\) (angle from dot product)

Let \(\vec{u},\vec{v}\in\mathbb{R}^n\) with \(\vec{v}\neq\vec{0}\).
Decompose \(\vec{u}\) relative to \(\vec{v}\): \[ \vec{u}=c_1\vec{e}+c_2\vec{n}, \] where \(\vec{e}=\frac{\vec{v}}{\left\lVert \vec{v} \right\rVert}\) is the unit vector in the direction of \(\vec{v}\) and \(\vec{n}\perp\vec{v}\) in the plane \(W=\mathop{\mathrm{span}}\{\vec{u},\vec{v}\}\).
The scalar \(c_1\) measures the component of \(\vec{u}\) along \(\vec{v}\).
The vector \(c_1\vec{e}\) is the projection of \(\vec{u}\) onto \(\vec{v}\), denoted by \(\operatorname{proj}_{\vec{v}}\vec{u}\).

Start with \(\vec{u}=c_1\vec{e}+c_2\vec{n}\).
Take the dot product with \(\vec{v}\): \[ \vec{u}\!\cdot\!\vec{v} = c_1(\vec{e}\!\cdot\!\vec{v}) = c_1\left\lVert \vec{v} \right\rVert. \]
Hence \[ c_1=\frac{\vec{u}\!\cdot\!\vec{v}}{\left\lVert \vec{v} \right\rVert}. \]
Therefore, \[ \operatorname{proj}_{\vec{v}}\vec{u} = c_1\vec{e} = \frac{\vec{u}\!\cdot\!\vec{v}}{\left\lVert \vec{v} \right\rVert^2}\,\vec{v} = \frac{\vec{u}\!\cdot\!\vec{v}}{\vec{v}\!\cdot\!\vec{v}}\,\vec{v}. \]

Let \(\vec{u}=(1,0,-1)\) and \(\vec{v}=(-2,2,1)\).
Compute \(\vec{u}\!\cdot\!\vec{v}\) and \(\left\lVert \vec{v} \right\rVert\). Answer two numbers on iClicker separated by a comma.
Is the angle between \(\vec{u}\) and \(\vec{v}\) parallel, opposite, acute, obtuse, or perpendicular?

Scan the QR code or go to join.iclicker.com/MBNJ.

Let \(\vec{e}=\frac{\vec{v}}{\left\lVert \vec{v} \right\rVert}\) be the unit vector in the direction of \(\vec{v}\), and let \(\vec{n}\) be a unit vector perpendicular to \(\vec{v}\) in \(W=\mathop{\mathrm{span}}\{\vec{u},\vec{v}\}\).
Decompose \[ \vec{u}=c_1\vec{e}+c_2\vec{n}. \]
Take the dot product with \(\vec{v}\) and simplify the right-hand side.
Use your previous results to compute \(c_1\). Enter your answer on iClicker.

Scan the QR code or go to join.iclicker.com/MBNJ.

How do we define a “dot product” between \[ -1+x+x^2 \quad \text{and} \quad 3+3x \] in the space of polynomials?
Interpret them as coefficient vectors: \[ (-1,1,1) \quad \text{and} \quad (3,3,0). \]
Then \[ \langle -1,1,1 \rangle\!\cdot\!\langle 3,3,0 \rangle=0. \]
Even without a geometric angle, this value still has meaning.

In abstract vector spaces, we can still define a dot product (typically called an inner product).
However, we may no longer interpret it geometrically (no physical angle).
Instead, we interpret \(\vec{u}\!\cdot\!\vec{v}\) as measuring similarity or correlation between vectors.
In the polynomial example, the dot product is \(0\), meaning the vectors are uncorrelated (orthogonal).

Just as vector spaces are defined axiomatically, we define an inner product axiomatically.
For vectors \(\vec{u},\vec{v}\) in a vector space \(V\), an inner product is a function \(\left\langle \vec{u},\vec{v} \right\rangle : V\times V \to \mathbb{R}\) satisfying the following properties:
Linearity (in the first variable):
\(\left\langle a\vec{u}+b\vec{w},\vec{v} \right\rangle = a\left\langle \vec{u},\vec{v} \right\rangle + b\left\langle \vec{w},\vec{v} \right\rangle\) for all scalars \(a,b\).
Symmetry:
\(\left\langle \vec{u},\vec{v} \right\rangle = \left\langle \vec{v},\vec{u} \right\rangle\).
Positivity:
\(\left\langle \vec{u},\vec{u} \right\rangle \ge 0\) for all \(\vec{u}\in V\).
Definiteness:
\(\left\langle \vec{u},\vec{u} \right\rangle = 0\) if and only if \(\vec{u}=\vec{0}\).

What is a natural inner product between functions?
Think of a function \(f(x)\) as an infinite-dimensional vector, where each “coordinate” is indexed by \(x\).
For functions \(f(x)\) and \(g(x)\), we might try an infinite sum \(\sum_{x\in\mathbb{R}} f(x)g(x)\), but uncountable sums are not defined.
Instead, we replace the sum with an integral and define \[ \left\langle f,g \right\rangle = \int_{\mathbb{R}} f(x)g(x)\,dx. \]
It is straightforward to check that the above definition satisfies all the required axioms for inner products.

A vector space equipped with an inner product is called an inner product space.
In such spaces, \(\left\langle \vec{u},\vec{v} \right\rangle\) measures how two vectors are correlated.
If \(\left\langle \vec{u},\vec{v} \right\rangle=0\), we say \(\vec{u}\) and \(\vec{v}\) are orthogonal.
In this course, we mainly use the standard dot product on \(\mathbb{R}^n\) (and later the inner product of functions for Fourier decomposition).

We want a “better” basis for a vector space.
A basis is orthogonal if every pair of vectors has dot product \(0\).
A basis is orthonormal if it is orthogonal and each vector has length \(1\).
If a basis is orthogonal, we can obtain an orthonormal basis by normalizing each vector.
Given a linearly independent set, how can we systematically construct an orthogonal basis?
The Gram–Schmidt process gives an explicit algorithm to do this.

Let \(S=\{\vec{u},\vec{v}\}\) be linearly independent in \(\mathbb{R}^2\) and let \(W=\mathop{\mathrm{span}}\{\vec{u},\vec{v}\}\).
If \(S\) is not orthogonal, can we replace it with an orthogonal basis of \(W\)?
Idea: subtract the component that is parallel using orthogonal projection.

Start with linearly independent \(S=\{\vec{v}_1,\vec{v}_2\}\).
Set \(\vec{w}_1=\vec{v}_1\).
Remove the component of \(\vec{v}_2\) parallel to \(\vec{w}_1\): \[ \vec{w}_2=\vec{v}_2-\operatorname{proj}_{\vec{w}_1}\vec{v}_2. \]
Then \(\{\vec{w}_1,\vec{w}_2\}\) is an orthogonal basis of \(W\).
Normalize if desired to obtain an orthonormal basis.

Let \(S=\{\vec{v}_1,\dots,\vec{v}_k\}\) be linearly independent and let \(W=\mathop{\mathrm{span}}(S)\).
Set \(\vec{w}_1=\vec{v}_1\).
For \(j\ge2\), remove all components parallel to the previous vectors: \[ \vec{w}_j = \vec{v}_j - \sum_{i=1}^{j-1} \operatorname{proj}_{\vec{w}_i}\vec{v}_j. \]
Then \(\{\vec{w}_1,\dots,\vec{w}_k\}\) is an orthogonal basis of \(W\).
Normalize each \(\vec{w}_j\) if an orthonormal basis is needed.