Eigenspace

Eigenspace vs. matrix subspaces

Unlike the column, row, and null spaces, the eigenspace only exists for a square matrix. The eigenspace of a square matrix, $A$, is the set of vectors that preserve their direction under the linear transformation matrix, $A$. Furthermore, eigenspace can be complex in contrast to the other spaces of a matrix, which are always real.

Definition

Formally, an eigenspace of an $n\times n$ matrix, $A$, is defined as:

$$E_\lambda(A)=\{\bold{x}|A\bold{x=\lambda x}\}$$

, where $\lambda$ is an eigenvalue of $A$ and $\bold{x}$ is the corresponding eigenvector of $A$. Thus, eigenspace consists of all the eigenvectors of a matrix corresponding to an eigenvalue.

Example

Consider a matrix, $A=\begin{bmatrix}3 & 2\\3 & 4\end{bmatrix}$. Furthermore, consider an eigenvalue, $\lambda=6$, and the corresponding eigenvector, $\bold{x}=\begin{bmatrix}10\\15\end{bmatrix}$. Notice that all multiples of $\bold{x}$ (that is $span(\bold{x})$) would also make valid eigenvectors corresponding to the same eigenvalue, such as $\frac{1}{5}\bold{x}=\begin{bmatrix}2 \\ 3\end{bmatrix}$. Therefore, by definition, the eigenspace corresponding to eigenvalue 6 will have the span of $\bold{x}$.

Note: There are infinitely many eigenvectors corresponding to each eigenvalue of a matrix.

Below is a visualization that shows how eigenvectors of a matrix preserve their direction under the linear transformation of that matrix. The red and blue vectors are eigenvectors corresponding to the eigenvalue, $6$. These vectors kept their direction under the matrix A transformation. In contrast, a brown vector loses its direction.

Note: A matrix may have several distinct eigenvalues and an eigenspace corresponds to each of them.

Computing eigenvalues and eigenvectors in numpy

In numpy, we can compute eigenvalues and the basis of corresponding eigenspaces using the np.linalg.eig function. Let’s try it out for the matrix in the example above.

For the matrix $A$ in the example above, there are $2$ distinct eigenvalues and the corresponding eigenspaces. This isn’t, however, generalizable. A general $n\times n$ matrix $A$ may not have $n$ distinct eigenvalues. An eigenspace can have more than one dimension, but the sum of the dimensions of all the eigenspaces of $A$ can’t be larger than $n$.

Eigenspace vs. null space

The eigenvectors of a matrix, $A$, satisfy $A\bold{x=\lambda x}$ for some scalar $\lambda$. We can rewrite this as $(A-\lambda I)\bold{x=0}$, where $I$ is the identity matrix. In this way, we discover that the eigenspace of a matrix, $A$, corresponding to the eigenvalue, $\lambda$, is the null space of the matrix $A-\lambda I$.

Moreover, for a null space of $A-\lambda I$ to exist beyond zero-dimensional, $\lambda$ must force singularity on $A-\lambda I$. This implies

$$det(A-\lambda I)=0$$

Note: A matrix, $A$, may or may not be singular, but the matrix, $A-\lambda I$, is always singular, where $\lambda$ is an eigenvalue of $A$.

Eigenvalues computation

To compute the eigenvalues of a matrix, $A$, we need to solve the equation, $det(A-\lambda I)=0$ for $\lambda$. For each value of $\lambda$, a corresponding eigenvector, $\bold{x}$, can be found by solving $(A-\lambda I)\bold{x}=0$.

Example

The eigenvalues of $A=\begin{bmatrix}3 & 2\\3 & 4\end{bmatrix}$ can be computed as follows:

$$\begin{align*} &\begin{vmatrix}3-\lambda & 2\\3 & 4-\lambda\end{vmatrix}&=0 \\ &(3-\lambda)(4-\lambda)-6 &=0 \\ &\lambda^2-7\lambda+6&=0 \\ &\lambda^2-\lambda-6\lambda+6&=0 \\ &\lambda(\lambda-1)-6(\lambda-1)&=0 \\ &(\lambda-1)(\lambda-6)&=0 \\ &\lambda \in \{1,6\} \end{align*}$$

We can now use both values of $\lambda$ to calculate a corresponding eigenvector. Let’s find an eigenvector, $\bold{x}$, that corresponds to $\lambda=1$. To that end, we have to find the solution of $(A-\lambda I)\bold{x}=0$

$$\begin{align*} &\begin{bmatrix}2 & 2\\3 & 3\end{bmatrix} \begin{bmatrix}x_1 \\ x_2\end{bmatrix}= \begin{bmatrix}0 \\ 0\end{bmatrix}\\ &\left [\begin{array}{cc|c} 1 & 1 & 0 \\ 0 & 0 & 0 \end{array}\right ]\\ &\begin{bmatrix}x_1 \\ x_2\end{bmatrix}=\begin{bmatrix} -1 \\ 1\end{bmatrix} \end{align*}$$

Thus, $\bold{x}=\begin{bmatrix} -1 \\ 1\end{bmatrix}$ is an eigenvector corresponding to the eigenvalue of $1$. Furthermore, all multiples of $\bold{x}$ are also eigenvectors corresponding to the eigenvalue of $1$. All these eigenvectors make an eigenspace. Moreover, there’s another eigenspace corresponding to the eigenvalue of $6$.

Eigenspace of a singular matrix

In the case that a matrix, $A$, is singular, one of its eigenvalues must be equal to $0$. Furthermore, the null space of $A$ is the same as the eigenspace of $A$ corresponding to eigenvalue of $0$.

$$A\bold{x=0}\implies A\bold{x}=0\bold{x}$$

Properties of eigenvalues and eigenvectors

1. The sum of eigenvalues is the trace of the matrix.
2. The product of eigenvalues is the determinant of the matrix.

Note: That the product of eigenvalues is the determinant of the matrix corresponds to the statement that, for singular matrices, at least one eigenvalue equals zero.

3. The eigenspaces corresponding to different eigenvalues are linearly independent. That is, the union of the basis of all eigenspaces is a linearly-independent set.

4. The eigenvalues of a symmetric matrix are real.
5. The eigenspaces of a symmetric matrix are mutually orthogonal.

6. If $\lambda$ is an eigenvalue of $A$, then $\frac{1}{\lambda}$ will be an eigenvalue of $A^{-1}$ and the respective eigenspaces will be the same.

7. The eigenvalues of $A$ and $A^T$ are the same.

8. The eigenvalues of a triangular matrix are the diagonal entries of the matrix.