FAQ
Can the below example be fixed to work?
Thanks,
Alan Isaac

import multiprocessing as mp

class Test(object):
pass

def class_factory(x):
class ConcreteTest(Test):
_x = x
return ConcreteTest

def f(cls):
print cls._x

if __name__ == '__main__':
pool = mp.Pool(2)
pool.map(f, [class_factory(i) for i in range(4)])

Search Discussions

  • Robert Kern at Jan 28, 2011 at 7:23 pm

    On 1/28/11 1:02 PM, Alan wrote:
    Can the below example be fixed to work?
    Thanks,
    Alan Isaac

    import multiprocessing as mp

    class Test(object):
    pass

    def class_factory(x):
    class ConcreteTest(Test):
    _x = x
    return ConcreteTest

    def f(cls):
    print cls._x

    if __name__ == '__main__':
    pool = mp.Pool(2)
    pool.map(f, [class_factory(i) for i in range(4)])
    Send the (pickleable) factory and the arguments used to construct the instance,
    not the unpickleable instance itself.

    def g(factory, i):
    cls = factory(i)
    print cls._x

    if __name__ == '__main__':
    pool = mp.Pool(2)
    pool.map(g, zip([class_factory] * 4, range(4)))

    By the way, when asking for help like this, show us what your code did and
    describe what results you want. It can often be hard to figure out exactly what
    you mean by "work".

    --
    Robert Kern

    "I have come to believe that the whole world is an enigma, a harmless enigma
    that is made terrible by our own mad attempt to interpret it as though it had
    an underlying truth."
    -- Umberto Eco
  • Daniel Urban at Jan 28, 2011 at 7:25 pm

    On Fri, Jan 28, 2011 at 20:02, Alan wrote:
    Can the below example be fixed to work?
    Thanks,
    Alan Isaac

    import multiprocessing as mp

    class Test(object):
    ? ?pass

    def class_factory(x):
    ? ?class ConcreteTest(Test):
    ? ? ? ?_x = x
    ? ?return ConcreteTest

    def f(cls):
    ? ?print cls._x

    if __name__ == '__main__':
    ? ?pool = mp.Pool(2)
    ? ?pool.map(f, [class_factory(i) for i in range(4)])
    Only classes defined on the top level of a module are picklable (see
    http://docs.python.org/dev/py3k/library/pickle#what-can-be-pickled-and-unpickled
    ). The collections.namedtuple class factory function works around this
    limitation by setting the __module__ attribute of the created class,
    but I'm not sure if this solution can be used in this case.


    Daniel
  • Robert Kern at Jan 28, 2011 at 7:42 pm

    On 1/28/11 1:25 PM, Daniel Urban wrote:

    Only classes defined on the top level of a module are picklable (see
    http://docs.python.org/dev/py3k/library/pickle#what-can-be-pickled-and-unpickled
    ). The collections.namedtuple class factory function works around this
    limitation by setting the __module__ attribute of the created class,
    but I'm not sure if this solution can be used in this case.
    namedtuple's trick only works when you assign the created class to a name at the
    module level. E.g.

    MyFancyTuple = collections.namedtuple(...)

    The trick won't work for "anonymous" classes like the above use case.

    --
    Robert Kern

    "I have come to believe that the whole world is an enigma, a harmless enigma
    that is made terrible by our own mad attempt to interpret it as though it had
    an underlying truth."
    -- Umberto Eco
  • Hidura at Jan 28, 2011 at 7:34 pm
    What is the output?

    2011/1/28, Alan <alan.isaac at gmail.com>:
    Can the below example be fixed to work?
    Thanks,
    Alan Isaac

    import multiprocessing as mp

    class Test(object):
    pass

    def class_factory(x):
    class ConcreteTest(Test):
    _x = x
    return ConcreteTest

    def f(cls):
    print cls._x

    if __name__ == '__main__':
    pool = mp.Pool(2)
    pool.map(f, [class_factory(i) for i in range(4)])

    --
    http://mail.python.org/mailman/listinfo/python-list
    --
    Enviado desde mi dispositivo m?vil

    Diego I. Hidalgo D.
  • Alan at Jan 28, 2011 at 8:26 pm

    On Jan 28, 2:23 pm, Robert Kern wrote:
    Send the (pickleable) factory and the arguments used to construct the instance,
    not the unpickleable instance itself.

    def g(factory, i):
    cls = factory(i)
    print cls._x

    if __name__ == '__main__':
    pool = mp.Pool(2)
    pool.map(g, zip([class_factory] * 4, range(4)))


    If I change that to g((factory,i)) it does work.

    Thanks!

    Alan

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouppython-list @
categoriespython
postedJan 28, '11 at 7:02p
activeJan 28, '11 at 8:26p
posts6
users4
websitepython.org

People

Translate

site design / logo © 2022 Grokbase