Column Space

Definition

The column space of a matrix, $A$, often denoted by $C(A)$, is a vector space spanned by the column vectors of $A$.

The column-space of an $m \times n$ matrix $A$ is a subspace of $\R^m$. The dimensions of the column space are the number of linearly independent columns, that is, the rank of the matrix $A$.

Examples

1. The column space of $\begin{bmatrix}1 &0\\0&1\end{bmatrix}$ is $\R^2$.

2. The column space of $\begin{bmatrix}1 &2\\2&4\end{bmatrix}$ is a one-dimensional subspace of $\R^2$. This is because there’s a single linearly independent vector as $2\bold{c_1} =\bold{c_2}$. Any of these columns can be a basis for the subspace.

3. The column space of $\begin{bmatrix}0 &0\\0&0\end{bmatrix}$ is a zero-dimensional subspace of $\R^2$.

4. The column space of an $n \times n$ invertible matrix $A$ is $\R^n$. This is because all the columns of such a matrix are linearly independent. That is, $rank(A)=n$.

The columns of every $n \times n$ invertible matrix form a basis of $R^n$.

Basis of column space

A basis of the column space of a matrix, $A$, is a set of linearly independent columns of $A$. One algorithm for computing the basis is as follows:

1. Take the transpose of the matrix to convert the columns into rows.

2. Compute the $rref$ (reduced row echelon form) of the transposed matrix.

3. The set of non-zero rows in $rref$ is a linearly-independent set. These form the basis of the column space.

Note that the basis found by the algorithm above may not contain the original column vectors, but their linear combinations as elementary row operations have transformed the matrix into $rref$.

Let’s try it out by using the following code:

Verifying a basis of column space

Given a set, $B$, of linearly-independent vectors from $\R^m$, how do we know that the set is a basis for the column space of a given $m \times n$ matrix $A$? To do this, we need to determine the following:

1. Are all the columns of $A$ in the span of $B$? If they are, it will prove that $C(A)\subseteq span(B)$.

2. Are all the vectors in $B$ in the column space of $A$?. If they are, it will will prove that $span(B) \subseteq C(A)$.

Both points above together prove that $C(A)=span(B)$. Here’s one algorithm for this problem:

1. Create a matrix $C$ with columns that are the vectors in $B$.
2. Check whether the ranks of $A$ and $C$ are equal, say $r$.
3. Create a matrix, $D$, with columns of $A$ and $C$ (in any order).
4. If the rank of the matrix, $D$, is also $r$, then $B$ is a basis of $C(A)$.

The code below implements the algorithm above.

Column space and linear transformation

The multiplication of a matrix with a vector can be viewed as a linear transformation of the vector. Consider an $m \times n$ matrix $A$ and a vector, $\bold w \in \R^n$. If we look closely at the matrix, $A$, represented in column vectors as $A=\begin{bmatrix}\bold c_1 &\bold c_2&\cdots &\bold c_n\end{bmatrix}$ and the vector $\bold w$ in its individual components, namely $\bold w=\begin{bmatrix}w_1 &w_2&\cdots &w_n\end{bmatrix}^T$, then

$$A\bold w=w_1\bold c_1+w_2\bold c_2+\cdots+w_n\bold c_n \in C(A)$$

Note: A linear transformation matrix, $A$, maps every vector to the column space of $A$.

Examples

1. The column space of $A=\begin{bmatrix} 1 & 3\\-1 & -3\end{bmatrix}$ is a line through the origin in the direction of the vector, $\begin{bmatrix}1 &-1\end{bmatrix}^T$. If matrix $A$ is multiplied with any vector in $\R^2$, it maps the vector onto that line. Below is its visualization:

2. The column space of $A=\begin{bmatrix} 1 & 1 & -2 \\-2 & 1 & -2 \\0 & 1 & -2\end{bmatrix}$ is a plane through the origin in $\R^3$, with the first $2$ column vectors as the basis. The visualization below shows how all the vectors on a cube map onto the plane when transformed using $A$. This is true for every vector in $\R^3$.

3. The column space of $A=\begin{bmatrix} 1 & 2 & -1 \\-2 & 1 & 2 \\0 & 1 & 3\end{bmatrix}$ is $\R^3$, but the standard basisStandard basis of a vector space is the basis where each vector has exactly one non-zero value equalling 1. of $\R^3$ (represented by a $3 \times 3$ identity matrix) transforms to the columns of the matrix. The visualization below shows all the vectors on a cube transforming to other vectors but not collapsing to any plane or a line. This is also true for all the other vectors in $\R^3$.

A linear system, $A\bold x=\bold b$, is consistent if $\bold b \in C(A)$. For invertible coefficient matrices, every vector is in the column space. So, the system is always consistent and an exact solution can be obtained through the inverse of $A$. That is, $\bold x = A^{-1}\bold b$.