Trusted answers to developer questions
Trusted Answers to Developer Questions

Related Tags


What are Tensors?

Talha Ashar

Tensors are multi-dimensional arrays that store numerical data. Tensors can be thought of as a mathematical entity, similar to a generalized matrix, that interacts with other entities in a structure.

Types of tensors

The table below shows the different types of tensors and the names conventionally associated with each type:

Tensor Types

Unlike a matrix, the numerical data stored in the tensor shifts given a transformation in one of the entities it interacts with. As such, tensors always obey certain transformation rules.

Note: In deep learning, a 2-D Array is referred to as a matrix, and arrays with more than 22 dimensions are called tensors.



The main attributes of tensors are as follows:

  • Rank: The number of dimensions present within the tensor. For example, a 22-dimensional tensor has a rank of 22, while a 33-dimensional tensor has a rank of 33.

  • Axis: A specific dimension of a tensor. For example, a 22-dimensional tensor has 22 axes.

  • Shape: The length of each axis of a tensor. It is possible to reshape a tensor without disturbing any of the elements it contains.

Let’s take a look at each of these attributes with the help of an example. First, consider the tensor shown below:

t = [[1,2,3], [4,5,6], [7,8,9]]

The tensor t has 22 dimensions, so it has a rank of 22 and 22 possible axes.

The elements along the first axis ([1,2,3], [4,5,6], and [7,8,9]) represent arrays, whereas each value within these arrays (the second axis) represents the data.

Since the tensor t has 33 rows and 33 columns, the shape will be [3, 3].


Tensors provide great flexibility in their dimensions and shape, and therefore they are commonly used to store datasets that contain multi-dimensional information.

Some common applications are as follows:

  • 3-D Tensors: Used to store time-series data, e.g., medical scans. Each dimension in the tensor may correspond to frequency, time, and channels of signals, respectively. If data for multiple patients is needed, a 4-D tensor may be used, where the extra dimension represents the sample size.

  • 4-D Tensors: Used to store JPEG images. Each dimension may correspond to the sample size, height, width, and color depth of the image, respectively.

  • 5-D Tensors: Used to store video data. Each dimension may correspond to the sample size, frames, height, width, and color depth, respectively.


The example below shows how tensors are used in the Pytorch library:

import torch

# initialize tensor
a = torch.tensor([[1,2,3], [4,5,6], [7,8,9]])
b = torch.tensor([[1,2,3], [4,5,6], [7,8,9]], dtype=torch.int32)

# printing tensors
print("Tensor A: ", a)
print("Tensor B: ", b)

# extracting values
c = a[1][2]
d = a[2][1]
print("The extracted values are", c, "and", d)

e = c.item()
f = d.item()
print("The extracted values are", e, "and", f)


First, two tensor objects, a and b, are initialized using python lists in lines 4 and 5. For b, the tensor is initialized with a specific data type since the dtype attribute is provided during initialization.

You can extract a particular dimension of the tensor through python indexing. For example, in line 12, the tensor a is indexed for the value in the third column of its second row, i.e., a[1][2]. This indexing returns a tensor object, which can be converted into a simple numeric data type using the item function, as shown in lines 16 and 17.

Note: A comprehensive guide on tensors and their operations in Pytorch can be found in the Official Guide.




Talha Ashar
Copyright ©2022 Educative, Inc. All rights reserved

View all Courses

Keep Exploring