Singular Value Decomposition: SVD

Definition

The Singular Value Decomposition (SVD) of an $n\times d$ matrix, $A$, is the factorization of $A$, multiplied by the product of three matrices: $A = U\Sigma V^T$, where the matrices $U_{n\times n}$ and $V^T_{d\times d}$ are orthogonal and the matrix, $\Sigma_{n\times d}$, is a generalized diagonalGeneralizedDiagonal with non-negative real entries.

SVD in numpy

We can compute SVD of a matrix, A, using U,S,Vt = numpy.linalg.svd(A), where S is an array containing diagonal entries of $\Sigma$ and Vt is $V^T$. The diag function in the implementation below converts the S array into the generalized diagonal matrix, $\Sigma$. We’ve used the D variable to represent $\Sigma$ in the code.