Trusted answers to developer questions
Trusted Answers to Developer Questions

Related Tags

python
communitycreator
memory

How do you monitor the memory consumption of Python functions?

Harsh Jain

Why monitoring memory consumption is important

When we are developing an application that is going to be used by many users, we need some memory to execute that application at scale. When developing such an application, we usually don’t really care if the variables (i.e.,the memory occupied by them) are released to the operating system and unknowingly assigned new memory blocks. However, this can cause our application to crash due to the memory limit.

Hence, we should profile the memory consumption by using Python code to ensure that we are not creating unnecessary blocks of memory.

How to monitor memory consumption

We will be using the memory_profiler package; an open-source Python package that performs line-by-line memory profiling for your Python code.

Let’s first install the package by running the command below:

pip install memory_profiler

When you want to execute the Python file and perform memory profiling, you should use:

python -m memory_profiler <filename>.py

Now, let’s take a look at the code:

from memory_profiler import profile

@profile
def my_func():
    a = [1] * (10 ** 6)
    b = [2] * (2 * 10 ** 7)
    c = [3] * (2 * 10 ** 8)
    del b
    return a    
  
if __name__=='__main__':
  my_func()
Memory profiling of Python functions

Explanation:

  • In line 1, we import the required package.
  • In line 3, we add a decorator @profile to track the function line by line. You need to add this decorator before every function that you want to track for memory consumption.
  • In line 4, we define our function.
  • From lines 5-9, we define the logic of the function. This is just normal logic; you can add the logic as per your needs.
  • In line 12, we call our function.

When we run the above code with the statement:

python -m memory_profiler main.py

You can expect a similar output to the one shown below:

Output Image

So, in this way, we can track that our Python code efficiently uses the memory.

RELATED TAGS

python
communitycreator
memory
RELATED COURSES

View all Courses

Keep Exploring