FAQ
Hello all,

I have a service that runs in python 2.6.4. This service sends
LogRecords to a log monitoring app on my workstation running python
2.7. The LogRecord class is derived:

class LogRecord(logging.LogRecord):

def __init__(self, name, level, fn, lno, user, hostname, msg,
args, exc_info, func=None):

if sys.version_info[1] > 4:
logging.LogRecord.__init__(self, name, level, fn, lno,
msg, args, exc_info, func)
else:
logging.LogRecord.__init__(self, name, level, fn, lno,
msg, args, exc_info)

Now when I try to unpickle it:

record = cPickle.loads(data)

I get a TypeError exception:

TypeError: ('__init__() takes at least 8 arguments (1 given)', <class
'...gRecord'>, ())

I've searched the web and this group, but most results are old. It
worked when my workstation still ran python 2.6.

Thank you,

Ian.

Search Discussions

  • Peter Otten at Apr 27, 2011 at 4:41 pm

    ivdneut at gmail.com wrote:

    I have a service that runs in python 2.6.4. This service sends
    LogRecords to a log monitoring app on my workstation running python
    2.7. The LogRecord class is derived:

    class LogRecord(logging.LogRecord):

    def __init__(self, name, level, fn, lno, user, hostname, msg,
    args, exc_info, func=None):

    if sys.version_info[1] > 4:
    logging.LogRecord.__init__(self, name, level, fn, lno,
    msg, args, exc_info, func)
    else:
    logging.LogRecord.__init__(self, name, level, fn, lno,
    msg, args, exc_info)

    Now when I try to unpickle it:

    record = cPickle.loads(data)

    I get a TypeError exception:

    TypeError: ('__init__() takes at least 8 arguments (1 given)', <class
    '...gRecord'>, ())

    I've searched the web and this group, but most results are old. It
    worked when my workstation still ran python 2.6.
    The Problem is that as of Python 2.7 logging.LogRecord has become a newstyle
    class which is pickled/unpickled differently. I don't know if there is an
    official way to do the conversion, but here's what I've hacked up.
    The script can read pickles written with 2.6 in 2.7, but not the other way
    round.

    $ cat pickle_logrec.py
    import sys
    import pickle
    import logging

    class LogRecord(logging.LogRecord):

    def __init__(self, name, level, fn, lno, user, hostname, msg, args,
    exc_info, func=None):

    if sys.version_info[1] > 4:
    logging.LogRecord.__init__(self, name, level, fn, lno, msg,
    args, exc_info, func)
    else:
    logging.LogRecord.__init__(self, name, level, fn, lno, msg,
    args, exc_info)

    def makeLogRecord():
    return LogRecord(*[None]*9)

    if issubclass(LogRecord, object):
    print "LogRecord is a newstyle class"
    class MyUnpickler(pickle.Unpickler):
    def find_class(self, *args):
    klass = pickle.Unpickler.find_class(self, *args)
    if klass is LogRecord:
    return makeLogRecord
    return klass
    else:
    print "LogRecord is an oldstyle class"
    MyUnpickler = pickle.Unpickler

    if __name__ == "__main__":
    if "--load" in sys.argv:
    print "loading"
    with open("tmp.pickle") as f:
    restored = MyUnpickler(f).load()
    print restored
    else:
    print "dumping"
    with open("tmp.pickle", "w") as f:
    f.write(pickle.dumps(LogRecord("yadda", *[None]*8)))


    $ python2.6 pickle_logrec.py
    LogRecord is an oldstyle class
    dumping
    $ python2.6 pickle_logrec.py --load
    LogRecord is an oldstyle class
    loading
    <LogRecord: yadda, None, None, None, "None">
    $ python2.7 pickle_logrec.py --load
    LogRecord is a newstyle class
    loading
    <LogRecord: yadda, None, None, None, "None">

    No warranty, use at your own risk.
  • Vinay Sajip at Apr 27, 2011 at 8:39 pm

    On Apr 27, 5:41?pm, Peter Otten wrote:

    The Problem is that as of Python 2.7logging.LogRecord has become a newstyle
    class which is pickled/unpickled differently. I don't know if there is an
    official way to do the conversion, but here's what I've hacked up.
    The script can read pickles written with 2.6 in 2.7, but not the other way
    round.
    [code snipped]
    I don't know about "official", but another way of doing this is to
    pickle just the LogRecord's __dict__ and send that over the wire. The
    logging package contains a function makeLogRecord(d) where d is a
    dict.

    This is the approach used by the examples in the library documentation
    which pickle events for sending across a network:

    http://docs.python.org/howto/logging-cookbook.html#sending-and-receiving-logging-events-across-a-network

    The built-in SocketHandler pickles the LogRecord's __dict__ rather
    than the LogRecord itself, precisely because of the improved
    interoperability over pickling the instance directly.

    Regards,

    Vinay Sajip
  • Peter Otten at Apr 28, 2011 at 7:22 am

    Vinay Sajip wrote:
    On Apr 27, 5:41 pm, Peter Otten wrote:

    The Problem is that as of Python 2.7logging.LogRecord has become a
    newstyle class which is pickled/unpickled differently. I don't know if
    there is an official way to do the conversion, but here's what I've
    hacked up. The script can read pickles written with 2.6 in 2.7, but not
    the other way round.
    [code snipped]
    I don't know about "official", but another way of doing this is to
    pickle just the LogRecord's __dict__ and send that over the wire. The
    logging package contains a function makeLogRecord(d) where d is a
    dict.
    You are right, my approach is too complicated and only needed when the OP
    cannot modify the sending script -- which is unlikely.
    This is the approach used by the examples in the library documentation
    which pickle events for sending across a network:

    http://docs.python.org/howto/logging-cookbook.html#sending-and-receiving-
    logging-events-across-a-network
    The built-in SocketHandler pickles the LogRecord's __dict__ rather
    than the LogRecord itself, precisely because of the improved
    interoperability over pickling the instance directly.
    As a minimal change ensuring that the logging.LogRecord subclass used by the
    OP is a newstyle class in 2.6 with

    class LogRecord(logging.LogRecord, object):
    #...

    should work, too.
  • Ivdneut at Apr 29, 2011 at 8:36 am

    On Apr 28, 9:22?am, Peter Otten wrote:
    Vinay Sajip wrote:
    On Apr 27, 5:41 pm, Peter Otten wrote:

    The Problem is that as of Python 2.7logging.LogRecord has become a
    newstyle class which is pickled/unpickled differently. I don't know if
    there is an official way to do the conversion, but here's what I've
    hacked up. The script can read pickles written with 2.6 in 2.7, but not
    the other way round.
    [code snipped]
    I don't know about "official", but another way of doing this is to
    pickle just the LogRecord's __dict__ and send that over the wire. The
    logging package contains a function makeLogRecord(d) where d is a
    dict.
    You are right, my approach is too complicated and only needed when the OP
    cannot modify the sending script -- which is unlikely.
    This is the approach used by the examples in the library documentation
    which pickle events for sending across a network:
    http://docs.python.org/howto/logging-cookbook.html#sending-and-receiv...
    logging-events-across-a-network


    The built-in SocketHandler pickles the LogRecord's __dict__ rather
    than the LogRecord itself, precisely because of the improved
    interoperability over pickling the instance directly.
    As a minimal change ensuring that the logging.LogRecord subclass used by the
    OP is a newstyle class in 2.6 with

    class LogRecord(logging.LogRecord, object):
    ? ? #...

    should work, too.
    I tried this, but it didn't work. Pickling the __dict__ and then use
    makeLogRecord does the trick.

    Thank you very much for the excellent help,

    Ian.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouppython-list @
categoriespython
postedApr 27, '11 at 1:56p
activeApr 29, '11 at 8:36a
posts5
users3
websitepython.org

People

Translate

site design / logo © 2022 Grokbase