Search⌘ K
AI Features
Home
Courses
Linear Algebra for Data Science Using Python

Span, Basis, and Dimensions

Explore the fundamental concepts of span, basis, and dimensions within vector spaces to build a strong foundation in linear algebra. Understand how spanning sets generate vector spaces, learn what constitutes a basis with linearly independent vectors, and grasp the fixed nature of dimensions across bases. This lesson equips you with essential knowledge for applying these concepts to data science and Python coding projects.

We'll cover the following...

Span

The span of a set, UU, is another set, VV, consisting of all the linear combinations of vectors in UU. The set UU is referred to as a spanning set.

Examples

  1. span({[10],[01]})=R2span\bigg(\bigg\{\begin{bmatrix}1\\0\end{bmatrix},\begin{bmatrix}0\\1\end{bmatrix}\bigg\}\bigg)=\R^2

We can create any vector in R2\R^2, say [αβ]T\begin{bmatrix}\alpha & \beta\end{bmatrix}^T, using a linear combination of the spanning set X={[10]T,[10]T}X=\bigg\{\begin{bmatrix}1 & 0\end{bmatrix}^T, \begin{bmatrix}1 & 0\end{bmatrix}^T\bigg\}. So, XX spans R2\R^2. That is:

α[10]+β[01]=[αβ]\alpha\begin{bmatrix}1 \\ 0\end{bmatrix}+\beta\begin{bmatrix}0 \\ 1\end{bmatrix}=\begin{bmatrix}\alpha \\ \beta\end{bmatrix}

  1. span({[ab]})=span\bigg(\bigg\{\begin{bmatrix}a\\b\end{bmatrix}\bigg\}\bigg)= line in the direction of [ab]\begin{bmatrix}a\\b\end{bmatrix}

Note: A line consists of points that are also position-vectors.

Spanning set isn’t unique

A spanning set corresponding to a given span isn’t unique.

In a previous example, we showed that span({[10],[01]})=R2span\bigg(\bigg\{\begin{bmatrix}1\\0\end{bmatrix},\begin{bmatrix}0\\1\end{bmatrix}\bigg\}\bigg)=\R^2 . . However, there exists infinitely many spanning sets that span R2\R^2. For instance, span({[23],[57]})=R2span\bigg(\bigg\{\begin{bmatrix}2\\3\end{bmatrix},\begin{bmatrix}5\\7\end{bmatrix}\bigg\}\bigg)=\R^2. To prove this, we may show that [10]∈span({[23],[57]})\begin{bmatrix}1\\0\end{bmatrix}\in span\bigg(\bigg\{\begin{bmatrix}2\\3\end{bmatrix},\begin{bmatrix}5\\7\end{bmatrix}\bigg\}\bigg) and [01]∈span({[23],[57]})\begin{bmatrix}0\\1\end{bmatrix}\in span\bigg(\bigg\{\begin{bmatrix}2\\3\end{bmatrix},\begin{bmatrix}5\\7\end{bmatrix}\bigg\}\bigg). That is:

[10]=−7[23]+3[57]\begin{bmatrix}1\\0\end{bmatrix}=-7\begin{bmatrix}2\\3\end{bmatrix}+3\begin{bmatrix}5\\7\end{bmatrix}

[10]=5[23]−2[57]\begin{bmatrix}1\\0\end{bmatrix}=5\begin{bmatrix}2\\3\end{bmatrix}-2\begin{bmatrix}5\\7\end{bmatrix}

Note: A vector, v\bold v, is in the span of the vectors u1,u2,...,uk\bold{u_1,u_2,...,u_k} if v\bold{v} is a linear combination of the vectors u1,u2,...,uk\bold{u_1,u_2,...,u_k}, i.e., v=α1u1+α2u2+⋯+αkuk\bold v = \alpha_1\bold{u_1}+\alpha_2\bold{u_2}+\cdots+\alpha_k\bold{u_k}.

The linear combination test can be viewed as a consistency check of a linear system. In the animation below, the column vectors of the coefficient matrix are linearly independent causing the two by two matrix to be invertible. So, the linear system is consistent for every right-hand-side b.

Span results in a vector space

The span of any spanning set will create a vector space. To formally prove this, we have to show that all axioms of the vector space are satisfied with a span of a set. ​​Below, we prove only the closure axiom. Enthusiastic learners should try working out the other proofs too.

Consider U={u1,u2,⋯ ,uk}U=\{\bold{u_1},\bold{u_2},\cdots,\bold{u_k}\}, span(U)=Vspan(U)=V, vi,vj∈V\bold{v_i,v_j} \in V and that scalars are taken from field FF. We need to show that VV is closed under linear combinations.

γ1vi+γ2vj=γ1(α1u1+α2u2+⋯+αkuk)+γ2(β1u1+β2u2+⋯+βkuk)=(γ1α1+γ2β1)u1+(γ1α2+γ2β2)u2+⋯+(γ1αk+γ2βk)uk=α^1u1+α^2u2+⋯+α^kuk∈span(U)\begin{align*} \gamma_1\bold{v_i}+\gamma_2\bold{v_j}&=\gamma_1(\alpha_1\bold{u_1}+\alpha_2\bold{u_2}+\cdots+\alpha_k\bold{u_k})+\gamma_2(\beta_1\bold{u_1}+\beta_2\bold{u_2}+\cdots+\beta_k\bold{u_k})\\ &=(\gamma_1\alpha_1+\gamma_2\beta_1)\bold{u_1}+(\gamma_1\alpha_2+\gamma_2\beta_2)\bold{u_2}+\cdots+(\gamma_1\alpha_k+\gamma_2\beta_k)\bold{u_k}\\ &=\hat{\alpha}_1\bold{u_1}+\hat{\alpha}_2\bold{u_2}+\cdots+\hat{\alpha}_k\bold{u_k}\in span(U)\\ \end{align*}

Basis

A basis of a vector space, VV, is a set, UU, of linearly independent vectors such that span(U)=Vspan(U)=V. In other words, a basis is a spanning set with linearly independent vectors.

Examples

  1. span({[10],[01]})=R2span\bigg(\bigg\{\begin{bmatrix}1\\0\end{bmatrix},\begin{bmatrix}0\\1\end{bmatrix}\bigg\}\bigg)=\R^2, and the set U={[10],[01]}U=\bigg\{\begin{bmatrix}1\\0\end{bmatrix},\begin{bmatrix}0\\1\end{bmatrix}\bigg\} contains linearly independent vectors. So, UU is a basis for R2\R^2.
  2. span({[10],[01],[210],[−10.5]})=R2span\bigg(\bigg\{\begin{bmatrix}1\\0\end{bmatrix},\begin{bmatrix}0\\1\end{bmatrix},\begin{bmatrix}2\\10\end{bmatrix},\begin{bmatrix}-1\\0.5\end{bmatrix}\bigg\}\bigg)=\R^2, but the set H={[10],[01],[210],[−10.5]}H = \bigg\{\begin{bmatrix}1\\0\end{bmatrix},\begin{bmatrix}0\\1\end{bmatrix},\begin{bmatrix}2\\10\end{bmatrix},\begin{bmatrix}-1\\0.5\end{bmatrix}\bigg\} doesn’t contain linearly independent vectors, so, HH isn’t a basis for R2\R^2.

Note: The basis of a vector space is not unique. It’s not hard to see that the columns of every invertible d×dd \times d matrix form a basis for Rd\R^d.

Note: Basis vectors are also known as axes. This also implies that the axis of a space need not be orthogonal.

Equivalence of basis

Given two bases, U={u1,u2,⋯ ,un}U=\{\bold{u_1,u_2,\cdots,u_n}\} and Z={z1,z2,⋯ ,zm}Z=\{\bold{z_1,z_2,\cdots,z_m}\}, can we test if both span the same vector space? The answer is yes. We can check if every vector of one basis can be written as a linear combination of the vectors from the other basis or the other way around. Let’s write a code to do this.

Python 3.8
import numpy as np
from  numpy.linalg import matrix_rank as rank
# The spanner vectors are the columns of A
# Check if b a linear combination of columns of A?
def isLinearCombination(A, b): 
  # Make augmented matrix of the linear system
  Ag = np.hstack((A, b[:, np.newaxis]))
  if rank(Ag) == rank(A):
    return True
  return False
# Check for each vector in either set if it is a linear combination of the other.
def spanEachOther(U, Z):
  for i in range(Z.shape[1]):
    if not isLinearCombination(U, Z[:, i]):
      return False
  for j in range(U.shape[1]):
    if not isLinearCombination(Z, U[:, j]):
      return False
  return True
# Example span sets  
U = np.array([[1, 2, 4, 9], 
              [3, -1, 3, 7], 
              [1, 1, 0, 2]])
Z = np.array([[1, 2, 4, 9], 
              [-2, -1, 3, 7], 
              [1, 2, 4, 9]])
rs=['are NOT in the span of each other', 'are in the span of each other']
print(f'U and Z {rs[spanEachOther(U, Z)]}')

Note: If both UU and ZZ are linearly independent sets and are in each other’s span, then both UU and ZZ are bases of the same space, which means that the basis of space isn’t unique.

Dimensions of a vector space

The dimensions of a vector space are the number of basis vectors for that vector space. Although the bases for a vector space aren’t unique, the vector space dimensions are fixed, that is, every basis has the same number of vectors.

Examples

  1. Consider dim(Rd)=ddim(\R^d)=d. Every basis has dd independent vectors. The columns of a d×dd\times d identity matrix form the standard basis of Rd\R^d.
  2. Consider dim(span({[14−8]}))=1dim\Bigg(span\Bigg(\Bigg\{\begin{bmatrix}1\\4\\-8\end{bmatrix}\Bigg\}\Bigg)\Bigg)=1. This is a line through the origin in the direction of the spanning vector.