Search⌘ K
AI Features
Home
Courses
Python Concurrency for Senior Engineering Interviews

Pool Executors

Explore Python's concurrent.futures module focusing on pool executors. Understand how ThreadPoolExecutor and ProcessPoolExecutor facilitate parallel task execution with futures for efficient concurrency management. Learn how to use submit and map methods, handle blocking calls, manage thread and process pools, and handle task timeouts for practical concurrent programming.

We'll cover the following...

Pool Executors

In the previous sections, we studied how to create and manage threads and processes. However, managing these entities can be taxing on the developer so Python alleviates this burden by providing an interface which abstracts away the subtleties of starting and tearing down threads or processes. The concurrent.futures package provides the Executor interface which can be used to submit tasks to either threads or processes. The two subclasses are:

  • ThreadPoolExecutor

  • ProcessPoolExecutor

Tasks can be submitted synchronously or asynchronously to the pools for execution.

ThreadPoolExecutor

The ThreadPoolExecutor uses threads for executing submitted tasks. Let's look at a very simple example.

Using ThreadPoolExecutor

from concurrent.futures import ThreadPoolExecutor
from threading import current_thread


def say_hi(item):

    print("\nhi " + str(item) + " executed in thread id " + current_thread().name, flush=True)


if __name__ == '__main__':
    executor = ThreadPoolExecutor(max_workers=10)
    lst = list()
    for i in range(1, 10):
        lst.append(executor.submit(say_hi, "guest" + str(i)))

    for future in lst:
        future.result()

    executor.shutdown()
...