A Deep Dive into Subspaces, Vector Spaces, and Their Applications in Linear Algebra
What is a Subspace of R^n?
A set U of vectors in R^n is called a subspace of R^n if it satisfies the following properties:
- The zero vector 0-> ∈ U.
- If x-> ∈ U and y-> ∈ U, then x->+y-> ∈ U.
- If x-> ∈ U, then ax-> ∈ U for every real number a.
Any subspace of R^n other than {0} or R^n is called a proper subspace.
Why is a Plane Through the Origin a Subspace of R^3?
Every plane M through the origin in R^3 has equation ax + by + cz = 0 where a, b, and c are not all zero. Here
n-> = [] is a normal for the plane and M = {v in R^3 | n->. v-> = 0} where v-> = [] and n-> · v-> denotes the dot product. Then M is a subspace of R^3. Indeed we show that M satisfies S1, S2, and S3 as follows:
- 0 ∈ M because n->· 0-> = 0;
- If v-> ∈ M and v1-> ∈ M, then n->·(v->+v1->) = n->· v->+n->· v1->= 0+0 = 0, so v->+v1-> ∈ M;
- If v-> ∈ M, then n->·(av->) = a(n->· v->) = a(0) = 0, so av ∈ M.
Null Space and Image Space of a Matrix
What is the Null Space and the Image Space of an mxn Matrix A?
Subspaces can also be used to describe important features of an mxn matrix A. The null space of A, denoted nullA, and the image space of A, denoted imA, are defined by:
- null A = {x-> ∈ R^n | Ax-> = 0->}
- im A = {Ax-> | x-> ∈ R^n}.
Null A consists of all solutions x-> in R^n of the homogeneous system Ax->=0->. Im A is the set of all vectors y-> in R^m such that Ax->=y-> has a solution x->. Note that x-> is in nullA if it satisfies the condition Ax->=0->, while imA consists of vectors of the form Ax-> for some x-> in R^n.
Why is the Null Space a Subspace of R^n? Why is the Image Space a Subspace of R^m?
The null space and image space are subspaces because they satisfy the three properties of subspaces outlined earlier. You can verify this by applying the properties to the definitions of nullA and imA.
Span of a Set of Vectors
What is the Span of a Set of Vectors?
The set of all linear combinations of a set of vectors {x1->, x2->, …, xk->} is called the span of the x->, and is denoted:
span {x1->, x2->, …, xk->} = {t1x1-> +t2x2-> +···+tkxk-> | ti in R}.
If V=span {x1->, x2->, …, xk->}, we say that V is spanned by the vectors x1->, x2->, …, xk->, and that the vectors x1->, x2->, …, xk-> span the space V.
How Can Lines and Planes Be Described in Terms of Spanning Sets?
In particular, if v-> and w-> are two nonzero, nonparallel vectors in R^3, then M= span {v->, w->} is the plane in R^3 containing v-> and w->. Moreover, if d-> is any nonzero vector in R^3 (or R^2), then L=span {v->} = {td-> | t ∈ R} = Rd-> is the line with direction vector d->. Hence lines and planes can both be described in terms of spanning sets.
What Does the Span Theorem Say?
Let U=span {x1->, x2->, …, xk->} in R^n. Then:
- U is a subspace of R^n containing each xi->
- If W is a subspace of R^n and each xi-> ∈ W, then U ⊆ W (U is a subset of W)
Linear Independence
What Does it Mean to Say that a Set of Vectors is Independent?
We call a set {x1->, x2->, …, xk->} of vectors linearly independent (or simply independent) if it satisfies the following condition:
If t1x1-> +t2x2-> +···+tkxk-> = 0-> then t1 = t2 = ··· = tk = 0.
A set of vectors is independent if and only if the only linear combination that vanishes is the trivial one.
How Can We Tell Whether or Not a Given Set of Vectors is Independent?
To verify that a set {x1->, x2->, …, xk->} of vectors in R^n is independent, proceed as follows by using the independence test:
- Set a linear combination equal to zero: t1x1-> +t2x2-> +···+tkxk-> = 0->
- Show that ti=0 for each i (that is, the linear combination is trivial).
Of course, if some nontrivial linear combination vanishes, the vectors are not independent.
- Use an augmented matrix and make sure you have a leading 1 for all vector columns and 0’s to the right.
Why Can the Zero Vector Never Be an Element of Any Independent Set?
No set {x1->, x2->, …, xk->} of vectors containing the zero vector is independent because we have a vanishing nontrivial linear combination:
1 (0->)+0x1-> +0x2-> +···+0xk-> = 0->
Basis and Dimension
What Does the Fundamental Theorem Say?
Let U be a subspace of R^n. If U is spanned by m vectors, and if U contains k linearly independent vectors, then k
What Does it Mean to Say that a Set of Vectors is a Basis for a Subspace of R^n?
If U is a subspace of R^n, a set {x1->, x2->, …, xm->} of vectors in U is called a basis of U if it satisfies the following two conditions:
- {x1->, x2->, …, xm->} is linearly independent
- U = span {x1->, x2->, …, xm->}
What Does the Invariance Theorem Say?
If {x1->, x2->, …, xm->} and {y1->, y2->, …, xk->} are bases of a subspace U of R^n, then m=k
What is the Dimension of a Subspace in R^n?
If U is a subspace of R^n and {x1->, x2->, …, xm->} is any basis of U, the number, m, of vectors in the basis is called the dimension of U, denoted dimU=m.
Why Does it Make Sense to Say that dim{0->} = 0?
Returning to subspaces of R^n, we define dim{0->}=0. This amounts to saying {0->} has a basis containing no vectors. This makes sense because 0-> cannot belong to any independent set.
What Does Theorem 5.2.6 Say?
Let U/={0->} be a subspace of R^n. Then:
- U has a basis and dimU
- Any independent set in U can be enlarged (by adding vectors from any fixed basis of U) to a basis of U
- Any spanning set for U can be cut down (by deleting vectors) to a basis of U.
Column Space and Row Space
What is the Column Space of a Matrix? What is the Row Space?
The column space, colA, of A is the subspace of R^m spanned by the columns of A. The row space, rowA, of A is the subspace of R^n spanned by the rows of A.
What do Lemmas 5.4.1 and 5.4.2 Say?
Let A and B denote mxn matrices.
- If A—->B by elementary row operations, then rowA=rowB
- If A—–>B by elementary column operations, then colA=colB.
If R is a row-echelon matrix, then:
- The nonzero rows of R are a basis of rowR
- The columns of R containing leading ones are a basis of colR
What Does the Rank Theorem Say?
Let A denote any mxn matrix of rank r. Then dim(colA)=dim(rowA) = r. Moreover, if A is carried to a row echelon matrix R by row operations, then
- The r nonzero rows of R are a basis of rowA
- If the leading 1s lie in columns j1, j2, …., jr of R, then columns j1, j2, …, jr of A are a basis of colA
What Does Theorem 5.4.2 Say?
Let A denote an mxn matrix of rank r. Then
- The n-r basic solutions to the system Ax->=0-> provided by the gaussian algorithm are a basis of nullA, so dim[nullA]=n-r
- Theorem 5.4.1 provides a basis of imA = colA, and dim[imA]=r
Least Squares Approximation
What are the Normal Equations for a Vector?
The normal equations for a vector are (A^T A)z-> = A^T b->. This is a system of linear equations called the normal equations for z->.
What Does the Best Approximation Theorem Say?
Let A be an mxn matrix, let b-> be any column in R^m, and consider the system Ax->=b-> of m equations in n variables.
- Any solution z-> to the normal equations (A^T A) z->=A^T b-> is the best approximation to a solution to Ax->=b-> in the sense that IIb->-Az->II is the minimum value of IIb->-Ax->II as x-> ranges over all columns in R^n.
- If the columns of A are linearly independent, then A^T A is invertible and z-> is given uniquely by z->=(A^T A)^-1 A^T b->.
What is the Least Squares Approximating Line for a Set of Data? How Do We Find It?
Where di is the distance between the measured and observed point and called error at xi, the natural measure of how close the line y=f(x) is to the observed data points is the sum of squares S=d1^2 + d2^2 +….+dn^2 as the measure of error, and the line y=f(x) is to be chosen so as to make this sum as small as possible. This line is the least squares approximating line.
Suppose that n data points (x1, y1), (x2,y2), ….,(xn, yn) are given, where at least two of x1, x2, …, xn, are distinct. Put y->=[] and M=[]. Then the least squares approximating line for these data points has equation y=zo+z1x where z->=[] is found by gaussian elimination from the normal equations (M^T M) z-> = M^T y->. The condition that at least two of x1, x2, …., xn are distinct ensures that M^T M is an invertible matrix, so z-> is unique: z->= (M^T M)^-1 M^T y->
Vector Spaces
What is a Vector Space?
A vector space consists of a nonempty set V of objects (called vectors) that can be added, that can be multiplied by a real number (called a scalar in this context), and for which certain axioms hold. If v-> and w-> are two vectors in V, their sum is expressed as v->+w->, and the scalar product of v-> by a real number “a” is denoted as av->. These operations are called vector addition and scalar multiplication, respectively, and the ten axioms (5 for vector addition – 5 for scalar multiplication) are assumed to hold.
What Kinds of Objects Form Vector Spaces?
A vector space consists of a nonempty set V of objects (called vectors) that can be added, that can be multiplied by a real number (called a scalar in this context), and for which certain axioms hold. If v-> and w-> are two vectors in V, their sum is expressed as v->+w->, and the scalar product of v-> by a real number “a” is denoted as av->. These operations are called vector addition and scalar multiplication, respectively, and the ten axioms (5 for vector addition – 5 for scalar multiplication) are assumed to hold. The kinds of objects that form vector spaces must be closed under additional and scalar multiplication, which is true for real numbers themselves, mxn matrices, the sum of 2 matrices mxn, scalar multiples of mxn matrices, polynomials and geometric vectors.