Special Types of Matrices

Identity matrix

An identity matrix is a square matrix where values are equal to 1 on the diagonal of the matrix and 0 everywhere else. Mathematically, it can be shown as follows:

This would look like the following:

Here, $I\in R^{n\times n}.$

The identity matrix gives the following nice property when multiplied with another matrix $A$:

Square diagonal matrix

A square diagonal matrix is a more general case of the identity matrix, where the values along the diagonal can take any value and the off-diagonal values are zeros:

Tensors

An $n$-dimensional matrix is called a tensor. In other words, a matrix with an arbitrary number of dimensions is called a tensor. For example, a 4D tensor can be denoted as shown here:

Here, $R$ is the real number space.

Tensor/matrix operations

We’ll discuss the tensor or matrix operation one by one in detail.

Transpose

Transpose is an important operation defined for matrices or tensors. For a matrix, the transpose is defined as follows:

Here, $A^T$  denotes the transpose of $A$.

An example of the transpose operation can be illustrated as follows:

After the transpose operation:

For a tensor, transpose can be seen as permuting the dimensions’ order. For example, let’s define a tensor $S$, as shown here:

Now, one transpose operation (out of many) can be defined as follows:

Matrix multiplication

Matrix multiplication is another important operation that appears quite frequently in linear algebra.

Given the matrices $A\in R^{m\times n}$and $B \in R^{n \times p},$ the multiplication of $A$  and  $B$  is defined as follows:

Here, $C \in R^{m\times p}.$ ...