Multiprocessing
Explore Python's multiprocessing module to run CPU-bound tasks in parallel across multiple processor cores. Understand how separate processes work independently of the Global Interpreter Lock, improving performance for heavy computations. Learn the design patterns for subclassing Process, managing multiple processes, and the importance of the main guard in multiprocessing code.
Overview
Threads exist within a single OS process; that’s why they can share access to common objects. We can do concurrent computing at the process level, also. Unlike threads, separate processes cannot directly access variables set up by other processes. This independence is helpful because each process has its own GIL and its own private pool of resources. On a modern multi-core processor, a process may have its own core, permitting concurrent work with other cores.
The multiprocessing API was originally designed to mimic the threading API. However, the multiprocessing interface has evolved, and in recent versions of Python, it supports more features more robustly. The multiprocessing library is designed for when CPU-intensive jobs need to happen in parallel and multiple cores are available. Multiprocessing is not as useful when the processes spend a majority of ...