Search⌘ K
AI Features
Home
Courses
Python Concurrency for Senior Engineering Interviews

... continued

Explore the detailed use of the BaseManager register method in Python's multiprocessing. Understand parameters like proxytype, exposed, method_to_typeid, and create_method, and how they control shared objects and custom proxies across processes. Learn to create robust multiprocessing solutions with practical examples and grasp the impact of process start methods.

We'll cover the following...

The BaseManager's register method takes in a number of parameters and it's important that we discuss how they are used. We'll explain each one of them below:

  • typeid

  • callable

  • proxytype

  • exposed

  • method_to_typeid

  • create_method

In the previous sections, we already discussed the first two parameters. We'll study the remaining using an example. Consider the following Pair class that holds two values x and y.

class Pair:
    def __init__(self, x, y):
        self.x = x
        self.y = y

    def get_x(self):
        # return self.x
        return fancy_type(self.x)

    def set_x(self, new_x):
        self.x = new_x

    def set_y(self, new_y):
        self.y = new_y

Let's say we want to register an object of this class with a manager so that it can be shared among processes. The most straight-forward way to register would be:

pair = Pair(5, 6)
manager.register('get_pair', callable=lambda: pair)

All the other register parameters will take on default values. The proxy returned for the pair would be AutoProxy and all the public methods on the pair object are by default available for consuming processes.

If you wanted to restrict the methods available to consumers you can specify the list of available methods using the exposed parameter. The register call would be as follows:

pair = Pair(5, 6)
manager.register('get_pair', callable=lambda: pair, exposed=['get_x'])

The above change would only make the get_x() method available on the pair object. The below runnable script barfs when we try to access set_x() since it has not been exposed.

Python 3.5
from multiprocessing.managers import BaseManager
from multiprocessing import Process
import multiprocessing
import time
port_num = 55555
def process_task():
    manager = BaseManager(address=('', port_num))
    manager.register('get_pair')
    manager.connect()
    p = manager.get_pair()
    p.set_x(7)
    p.set_y(7)
    print(p.get_x())
    print(p.get_y())
class Pair:
    def __init__(self, x, y):
        self.x = x
        self.y = y
    def get_x(self):
        # return self.x
        return self.x
    def get_y(self):
        # return self.x
        return self.y
    def set_x(self, new_x):
        self.x = new_x
    def set_y(self, new_y):
        self.y = new_y
if __name__ == '__main__':
    p1 = Process(target=process_task)
    
    manager = BaseManager(address=('', port_num))
    pair = Pair(5, 6)
    manager.register('get_pair', callable=lambda: pair, exposed=['get_x'])
    manager.start()
    p1.start()
    time.sleep(3)
    manager.shutdown()

The next parameter is proxytype. We can ...