Tensors are multi-dimensional arrays that store numerical data. Tensors can be thought of as a mathematical entity, similar to a generalized matrix, that interacts with other entities in a structure.
The table below shows the different types of tensors and the names conventionally associated with each type:
Unlike a matrix, the numerical data stored in the tensor shifts given a transformation in one of the entities it interacts with. As such, tensors always obey certain transformation rules.
Note: In deep learning, a 2-D Array is referred to as a matrix, and arrays with more than $2$ dimensions are called tensors.
The main attributes of tensors are as follows:
Rank: The number of dimensions present within the tensor. For example, a $2$-dimensional tensor has a rank
of $2$, while a $3$-dimensional tensor has a rank
of $3$.
Axis: A specific dimension of a tensor. For example, a $2$-dimensional tensor has $2$ axes
.
Shape: The length of each axis
of a tensor. It is possible to reshape a tensor without disturbing any of the elements it contains.
Let’s take a look at each of these attributes with the help of an example. First, consider the tensor shown below:
t = [[1,2,3], [4,5,6], [7,8,9]]
The tensor t
has $2$ dimensions, so it has a rank
of $2$ and $2$ possible axes
.
The elements along the first axis
([1,2,3]
, [4,5,6]
, and [7,8,9]
) represent arrays, whereas each value within these arrays (the second axis
) represents the data.
Since the tensor t
has $3$ rows and $3$ columns, the shape will be [3, 3]
.
Tensors provide great flexibility in their dimensions and shape, and therefore they are commonly used to store datasets that contain multi-dimensional information.
Some common applications are as follows:
3-D Tensors: Used to store time-series data, e.g., medical scans. Each dimension in the tensor may correspond to frequency, time, and channels of signals, respectively. If data for multiple patients is needed, a 4-D tensor may be used, where the extra dimension represents the sample size.
4-D Tensors: Used to store JPEG images. Each dimension may correspond to the sample size, height, width, and color depth of the image, respectively.
5-D Tensors: Used to store video data. Each dimension may correspond to the sample size, frames, height, width, and color depth, respectively.
The example below shows how tensors are used in the Pytorch
library:
import torch# initialize tensora = torch.tensor([[1,2,3], [4,5,6], [7,8,9]])b = torch.tensor([[1,2,3], [4,5,6], [7,8,9]], dtype=torch.int32)# printing tensorsprint("Tensor A: ", a)print("Tensor B: ", b)# extracting valuesc = a[1][2]d = a[2][1]print("The extracted values are", c, "and", d)e = c.item()f = d.item()print("The extracted values are", e, "and", f)
First, two tensor objects, a
and b
, are initialized using python
lists in lines 4 and 5. For b
, the tensor is initialized with a specific data type since the dtype
attribute is provided during initialization.
You can extract a particular dimension of the tensor through python
indexing. For example, in line 12, the tensor a
is indexed for the value in the third column of its second row, i.e., a[1][2]
. This indexing returns a tensor object, which can be converted into a simple numeric data type using the item
function, as shown in lines 16 and 17.
Note: A comprehensive guide on tensors and their operations in
Pytorch
can be found in the Official Guide.