Search⌘ K
AI Features
Home
Courses
Linear Algebra for Data Science Using Python

Inverse of a Rectangular Matrix

Explore the concept of matrix invertibility extended to rectangular matrices. Understand the criteria involving rank for the existence of left or right inverses, and learn how to compute these inverses using matrix multiplication and transposes. This lesson helps you apply these principles in data science tasks involving non-square matrices.

We'll cover the following...

We’ve already discussed the inverse of a square matrix in detail and provided multiple examples. In this lesson, we’ll extend the discussion to rectangular matrices, which are matrices that have a different number of columns and rows.

Rank and invertibility

The rank of a matrix generally answers the question of invertibility. An m×nm \times n matrix AA is invertible if it has either a full column rank or a full row rank.

r(A)=min(m,n)r(A) = min(m,n)

Also, for any matrix, AA,

r(A)=r(ATA)=r(AAT)r(A)=r(A^TA)=r(AA^T)

Python 3.8
import numpy as np
from numpy.linalg import matrix_rank as r
order = np.random.randint(low=2, high=10, size=2)
m, n = order[0], order[1]
A = np.random.rand(m, n)
print(f'order(A)={A.shape}\nr(A)={r(A)}\nr(ATA)={r(A.T.dot(A))}\nr(AAT)={r(A.dot(A.T))}')

For square matrices, the number of rows, mm, is equal to the number of columns, nn. This allows square matrices to have a two-sided inversetwosided_Inverse. For rectangular matrices, the condition doesn’t hold. They either have a left inverse or a right inverse.

Left inverse

If a matrix, AA, has full column rank, r(A)=nr(A)=n, then it would have a left inverse, LL, such that:

A matrix with a full column rank could have multiple left inverses.

One possibility is:

L=(ATA)−1ATL = (A^T A)^{-1} A^T

For a full column rank matrix, Am×nA_{m \times n}, say, r(A)=nr(A)=n, the matrix, (ATA)(A^T A), will always be square and invertible. We can prove that LL will be a left inverse of AA as follows:

r(A)=norder(ATA)=n×nr(ATA)=n(ATA)−1(ATA)=I((ATA)−1AT)A=ILA=I\begin{align*} &r(A) &= n \\ &order(A^TA) &= n\times n \\ &r(A^TA) &= n \\ &(A^TA)^{-1}(A^TA) &= I \\ &((A^TA)^{-1}A^T)A &= I \\ &LA &= I \end{align*}

Python 3.8
import numpy as np
from numpy.linalg import inv, matrix_rank as r
m, n = 4, 3
A = np.random.rand(m, n)
# Compute and print left inverse if it exists
if r(A) == A.shape[1]: #check if there is a full column rank
  L = inv(A.T.dot(A)).dot(A.T)
  LA = L.dot(A)
  LA[np.abs(LA)<0.00000000001] = 0
  print(np.round(LA).astype(int))

Right inverse

If a matrix, AA, has a full row rank r(A)=mr(A)=ma then it would have a right inverse, RR, such that:

A matrix with a full row rank could have multiple right inverses.

One possibility is:

R=AT(AAT)−1R = A^T(AA^T)^{-1}

For a full row rank matrix, AA, the matrix AATAA^T will always be square and invertible. We can prove that RR will be right inverse of AA as follows:

r(A)=morder(AAT)=m×mr(AAT)=m(AAT)(AAT)−1=IA(AT(AAT)−1)=IAR=I\begin{align*} &r(A) &= m \\ &order(AA^T) &= m\times m \\ &r(AA^T) &= m \\ &(AA^T)(AA^T)^{-1} &= I \\ &A(A^T(AA^T)^{-1}) &= I \\ &AR &= I \end{align*}

Python 3.8
import numpy as np
from numpy.linalg import inv, matrix_rank as r
m, n = 3, 4
A = np.random.rand(m, n)
# Compute and print right inverse if it exists
if r(A) == A.shape[0]: #check if there is a full row rank
  R = A.T.dot(inv(A.dot(A.T)))
  AR = A.dot(R)
  AR[np.abs(AR)<0.00000000001] = 0
  print(np.round(AR).astype(int))