We said before that defining an inner product gives us a norm – a notion of magnitude.
For , if the dot product is used as the inner product, then , our common notion of Euclidean distance from the origin.
We can also extend this to find the distance from any other vector (not just the origin) by computing .
However, when dealing with abstract vector spaces like , we also get these notions of magnitude and distance – how "big" a function is and how "far" apart two functions are – even if it’s not intuitive at first.
By using as the inner product of two functions in , the distance between them is now . Now that we have a notion of distance between two functions, we can treat these functions as vectors and perform calculus on them – this is called functional analysis.
Definition. Two vectors are orthogonal if . (You can visualize this as them being perpendicular to each other in the plane that contains them and the origin.)
For example, if you think of the standard basis of , you can see that any two vectors in this set will be orthogonal to each other.
Also, every vector is orthogonal to the zero vector.
Definition. Let be a subspace of . A vector in is said to be orthogonal to if is orthogonal to every in . The collection of all vectors orthogonal to is denoted by , which is called the orthogonal complement of .
If , then contains only the zero vector.
Theorem. Let be an matrix. The orthogonal complement of is , and the orthogonal complement of is .
Theorem. If is spanned by , and is orthogonal to , then is in .
Ex. If and , then is in because it is orthogonal to both of those vectors.
Definition. A set of vectors is said to be orthogonal if each distinct pair of vectors in the set are orthogonal to each other (and .)
Theorem. If is an orthogonal set of vectors, then is a basis for (i.e. are linearly independent).
Definition. An orthogonal basis for a subspace of is a basis for that is also an orthogonal set.
It turns out that it is easier to calculate the coordinates of points in an orthogonal basis than those in a non-orthogonal basis.
Theorem. Let be an orthogonal basis for a subspace of . For each in , the weights in the linear combination are given by .