Python Multiprocessing Pickle Protocol

I am using the Python multiprocessing module to place objects onto a queue and have them processed by several workers. My first issue was getting bound instance methods to pickle,

Solution 1:

If it's not possible to change the pickle protocol the multiprocessing package uses, then define __getstate__ and __setstate__ for your objects:

import pickle

class Foo(object):
    __slots__ = ['this', 'that', 'other']

    def __init__(self):
        self.this = 1
        self.that = 2
        self.other = 3

    def __getstate__(self):
        return dict((name, getattr(self, name))
                    for name in self.__slots__)

    def __setstate__(self, state):
        for name, value in state.items():
            setattr(self, name, value)

pickle.dumps(Foo(), protocol=0)