FAQ
Hey guys,

After a brief discussion with Pierre I'm taking this discussion to here.
We have the igbinary pecl extension available [1], however not many of our
userbase are familiar that it even exists.
Here [2] is the performance boost we get from using igbinary with
serialize() and unserialize().

Here we get a great perf boost, and what's more excellent as we don't have
to break BC to get there!

This seems like a no-brainer and a good investment in the speed increase of
PHP by default for the future users of the platform.

[1] http://pecl.php.net/package/igbinary
[2] http://www.php.net/~pierre/vcqa/apcigninary_perf.png

Comments please.

Regards,
Paul Dragoonis.

Search Discussions

  • Arpad Ray at Aug 18, 2011 at 11:30 am
    Hi,
    On Thu, Aug 18, 2011 at 12:12 PM, Paul Dragoonis wrote:
    Hey guys,

    After a brief discussion with Pierre I'm taking this discussion to here.
    We have the igbinary pecl extension available [1], however not many of our
    userbase are familiar that it even exists.
    Here [2] is the performance boost we get from using igbinary with
    serialize() and unserialize().

    Here we get a great perf boost, and what's more excellent as we don't have
    to break BC to get there!
    It would break BC for anyone with serialized strings stored in the
    current format.

    Maybe bundle it in core and provide an ini setting like
    session.serialize_handler, but I don't think making it the default in
    5.4 is an option.

    Regards,

    Arpad
    This seems like a no-brainer and a good investment in the speed increase of
    PHP by default for the future users of the platform.

    [1] http://pecl.php.net/package/igbinary
    [2] http://www.php.net/~pierre/vcqa/apcigninary_perf.png

    Comments please.

    Regards,
    Paul Dragoonis.
  • Roger Llopart Pla at Aug 18, 2011 at 11:38 am
    I agree with bundling it in the core aswell, but having it as default, as
    stated, would be a huge BC break.

    2011/8/18 Arpad Ray <arraypad@gmail.com>
    Hi,
    On Thu, Aug 18, 2011 at 12:12 PM, Paul Dragoonis wrote:
    Hey guys,

    After a brief discussion with Pierre I'm taking this discussion to here.
    We have the igbinary pecl extension available [1], however not many of our
    userbase are familiar that it even exists.
    Here [2] is the performance boost we get from using igbinary with
    serialize() and unserialize().

    Here we get a great perf boost, and what's more excellent as we don't have
    to break BC to get there!
    It would break BC for anyone with serialized strings stored in the
    current format.

    Maybe bundle it in core and provide an ini setting like
    session.serialize_handler, but I don't think making it the default in
    5.4 is an option.

    Regards,

    Arpad
    This seems like a no-brainer and a good investment in the speed increase of
    PHP by default for the future users of the platform.

    [1] http://pecl.php.net/package/igbinary
    [2] http://www.php.net/~pierre/vcqa/apcigninary_perf.png

    Comments please.

    Regards,
    Paul Dragoonis.
    --
    PHP Internals - PHP Runtime Development Mailing List
    To unsubscribe, visit: http://www.php.net/unsub.php
  • Paul Dragoonis at Aug 18, 2011 at 11:48 am

    On Thu, Aug 18, 2011 at 12:38 PM, Roger Llopart Pla wrote:

    I agree with bundling it in the core aswell, but having it as default, as
    stated, would be a huge BC break.

    2011/8/18 Arpad Ray <arraypad@gmail.com>
    Hi,

    On Thu, Aug 18, 2011 at 12:12 PM, Paul Dragoonis <dragoonis@gmail.com>
    wrote:
    Hey guys,

    After a brief discussion with Pierre I'm taking this discussion to here.
    We have the igbinary pecl extension available [1], however not many of our
    userbase are familiar that it even exists.
    Here [2] is the performance boost we get from using igbinary with
    serialize() and unserialize().

    Here we get a great perf boost, and what's more excellent as we don't have
    to break BC to get there!
    It would break BC for anyone with serialized strings stored in the
    current format.

    Maybe bundle it in core and provide an ini setting like
    session.serialize_handler, but I don't think making it the default in
    5.4 is an option.
    Sorry guys, I didn't realise there was an actual BC break in the serialized
    structure.
    I agree with bundling and providing a .ini setting to activate, but off by
    default.

    Regards,

    Arpad
    This seems like a no-brainer and a good investment in the speed increase of
    PHP by default for the future users of the platform.

    [1] http://pecl.php.net/package/igbinary
    [2] http://www.php.net/~pierre/vcqa/apcigninary_perf.png

    Comments please.

    Regards,
    Paul Dragoonis.
    --
    PHP Internals - PHP Runtime Development Mailing List
    To unsubscribe, visit: http://www.php.net/unsub.php
  • Kiall Mac Innes at Aug 18, 2011 at 12:25 pm
    Some others systems use something similar to syntax checking (ldap password
    hashes).

    Baisically, prefix a code to the serialized string to indicate what method
    was used to serialize.

    Eg no prefix, or $O$ for the old style.

    Or $I$ for the igbinary syntax.

    This has the advantage of allowing additional formats to be added over time
    without breaking BC.

    Kiall
    On Aug 18, 2011 12:49 p.m., "Paul Dragoonis" wrote:
    On Thu, Aug 18, 2011 at 12:38 PM, Roger Llopart Pla wrote:

    I agree with bundling it in the core aswell, but having it as default, as
    stated, would be a huge BC break.

    2011/8/18 Arpad Ray <arraypad@gmail.com>
    Hi,

    On Thu, Aug 18, 2011 at 12:12 PM, Paul Dragoonis <dragoonis@gmail.com>
    wrote:
    Hey guys,

    After a brief discussion with Pierre I'm taking this discussion to
    here.
    We have the igbinary pecl extension available [1], however not many of our
    userbase are familiar that it even exists.
    Here [2] is the performance boost we get from using igbinary with
    serialize() and unserialize().

    Here we get a great perf boost, and what's more excellent as we don't have
    to break BC to get there!
    It would break BC for anyone with serialized strings stored in the
    current format.

    Maybe bundle it in core and provide an ini setting like
    session.serialize_handler, but I don't think making it the default in
    5.4 is an option.
    Sorry guys, I didn't realise there was an actual BC break in the
    serialized
    structure.
    I agree with bundling and providing a .ini setting to activate, but off by
    default.

    Regards,

    Arpad
    This seems like a no-brainer and a good investment in the speed
    increase
    of
    PHP by default for the future users of the platform.

    [1] http://pecl.php.net/package/igbinary
    [2] http://www.php.net/~pierre/vcqa/apcigninary_perf.png

    Comments please.

    Regards,
    Paul Dragoonis.
    --
    PHP Internals - PHP Runtime Development Mailing List
    To unsubscribe, visit: http://www.php.net/unsub.php
  • Arvids Godjuks at Aug 18, 2011 at 1:25 pm
    Just add a check on what format is used on read, write with the new
    format. Deprecate the old format in next version after 5.4 (or drop it
    at all in 6.0 or whatever the new major version will be).

    Make a noise about the change (I can do that for the Russian speaking
    community with ease).
    It will be transparent.

    People using serialized caches can just drop them and re-init. People
    storing data can recode it with ease - just read it and write it back
    - it will be in new format.

    Problem solved :)
  • Johannes Schlüter at Aug 18, 2011 at 1:16 pm

    On Thu, 2011-08-18 at 12:48 +0100, Paul Dragoonis wrote:
    I agree with bundling and providing a .ini setting to activate, but off by
    default.
    An ini setting is evil for this. People use serialized data as transport
    format having that depend on an ini setting is bad.

    suggestions:
    - The decode function could check the format ("old" format will always
    begin by one of aisfb or so followed by a colon, igbinary something
    else)
    - serialize() might get an extra parameter to set the format
    - lateron, if we want: if old format is being use a deprecation notice
    is shown when the older fomrat is being used)

    johannes
  • Ferenc Kovacs at Aug 18, 2011 at 1:33 pm

    2011/8/18 Johannes Schlüter <johannes@schlueters.de>:
    On Thu, 2011-08-18 at 12:48 +0100, Paul Dragoonis wrote:
    I agree with bundling and providing a .ini setting to activate, but off by
    default.
    An ini setting is evil for this. People use serialized data as transport
    format having that depend on an ini setting is bad.
    we already do this (session.serialize_handler, defaults to our own
    format, but can be set to wddx or igbinary)
    the userland session handler should be handler agnostic, it shouldn't
    really break the application if the serialization format change, as
    the handler should know or care whats inside the $data as long as it
    can be passed around.
    but imo this sucks. your handler cannot look into the data, the only
    available methods for packing and unpacking the session data can only
    used to read and write the $_SESSION superglobal, so for example if
    you to load another session for some reason, you have to save the
    value of the $_SESSION, empty $_SESSION, load the encoded session,
    call session_decode, fetch the $_SESSION then restore it from your
    copy.
    and I really hate that the php session format is almost compatible
    with serialize/unserialize. if I remember correctly only lacks the
    a:number_of_items_in_the_session:{} around the session data.
    why did we did this? to save a few bytes? /o\
    sorry for the rant.

    --
    Ferenc Kovács
    @Tyr43l - http://tyrael.hu
  • Johannes Schlüter at Aug 18, 2011 at 1:42 pm

    On Thu, 2011-08-18 at 15:33 +0200, Ferenc Kovacs wrote:
    we already do this (session.serialize_handler, defaults to our own
    format, but can be set to wddx or igbinary)
    Well the session handler doesn't matter too much and is more of an admin
    thing, unlike serialize()/unserialize(). Those are the ones which
    shouldn't be set by configuration as they are used programmatically.

    The format is relevant for session though, too, as people use memcache
    or something to share sessions from multiple hosts.
    and I really hate that the php session format is almost compatible
    with serialize/unserialize. if I remember correctly only lacks the
    a:number_of_items_in_the_session:{} around the session data.
    why did we did this? to save a few bytes? /o\
    History with register globals, back then each variable had to be
    serialized individually based on the list of registered vars. We
    resisted from changing the format to keep compatibility. (some people
    also want to stay sessions alive while updating)

    johannes
  • Ferenc Kovacs at Aug 18, 2011 at 1:50 pm

    2011/8/18 Johannes Schlüter <johannes@schlueters.de>:
    On Thu, 2011-08-18 at 15:33 +0200, Ferenc Kovacs wrote:
    we already do this (session.serialize_handler, defaults to our own
    format, but can be set to wddx or igbinary)
    Well the session handler doesn't matter too much and is more of an admin
    thing, unlike serialize()/unserialize(). Those are the ones which
    shouldn't be set by configuration as they are used programmatically.
    you are right, sorry for the offtopic.

    --
    Ferenc Kovács
    @Tyr43l - http://tyrael.hu
  • Ferenc Kovacs at Aug 18, 2011 at 11:48 am

    On Thu, Aug 18, 2011 at 1:30 PM, Arpad Ray wrote:
    Hi,
    On Thu, Aug 18, 2011 at 12:12 PM, Paul Dragoonis wrote:
    Hey guys,

    After a brief discussion with Pierre I'm taking this discussion to here.
    We have the igbinary pecl extension available [1], however not many of our
    userbase are familiar that it even exists.
    Here [2] is the performance boost we get from using igbinary with
    serialize() and unserialize().

    Here we get a great perf boost, and what's more excellent as we don't have
    to break BC to get there!
    It would break BC for anyone with serialized strings stored in the
    current format.

    Maybe bundle it in core and provide an ini setting like
    session.serialize_handler, but I don't think making it the default in
    5.4 is an option.
    agree with the BC part, just wanted to mention that that igbinary can
    already be set as a session handler via session.serialize_handler if
    the extension present.

    --
    Ferenc Kovács
    @Tyr43l - http://tyrael.hu
  • Kingsquare.nl - Robin Speekenbrink at Aug 18, 2011 at 12:02 pm
    Hello everyone,

    My first post after watching the list for a while: my opionion on this
    would also be incorporate it into the core but with BC compatibility:
    the serializer could (easily from what i understand) check to see if the
    serialized format is in the `old` format and read that accordingly, but
    write it back in the new... That way the problem would 'clear' itself up.. ?

    (or is this just simple 'user'-level thinking? :P)

    With kind regards,

    Robin Speekenbrink
    Kingsquare Information Services

    Op 18-8-2011 13:48, Ferenc Kovacs schreef:
    On Thu, Aug 18, 2011 at 1:30 PM, Arpad Raywrote:
    Hi,

    On Thu, Aug 18, 2011 at 12:12 PM, Paul Dragooniswrote:
    Hey guys,

    After a brief discussion with Pierre I'm taking this discussion to here.
    We have the igbinary pecl extension available [1], however not many of our
    userbase are familiar that it even exists.
    Here [2] is the performance boost we get from using igbinary with
    serialize() and unserialize().

    Here we get a great perf boost, and what's more excellent as we don't have
    to break BC to get there!
    It would break BC for anyone with serialized strings stored in the
    current format.

    Maybe bundle it in core and provide an ini setting like
    session.serialize_handler, but I don't think making it the default in
    5.4 is an option.
    agree with the BC part, just wanted to mention that that igbinary can
    already be set as a session handler via session.serialize_handler if
    the extension present.
  • Pierre Joye at Aug 18, 2011 at 12:22 pm
    hi!
    On Thu, Aug 18, 2011 at 1:12 PM, Paul Dragoonis wrote:

    After a brief discussion with Pierre I'm taking this discussion to here.
    For the record here, I'm not in favor of bundling igbinary (or other
    custom serializer). My main concerns are:

    . the PHP serializer can be overridden at runtime (as in after the build)
    . there will always have some new faster, better, sexier serializer,
    depending on the app/usages
    . we can't enable it by default (as other stated too here) for BC
    reason (and interop)
    . not enabling it by default defeats the purpose of bundling it
    . easier to support it while being only in pecl ('do you use the core
    version or latest pecl?")


    Cheers,
  • Stas Malyshev at Aug 18, 2011 at 5:27 pm
    Hi!
    On 8/18/11 5:22 AM, Pierre Joye wrote:
    . not enabling it by default defeats the purpose of bundling it
    Actually, I don't think it does. It's one thing to tell people "put this
    option into php.ini" and quite another is "download it from PECL (?),
    compile it (?!) and install it". The first is within the limits of any
    literate person, the second is sysadmin territory.
    . easier to support it while being only in pecl ('do you use the core
    version or latest pecl?")
    It is a concern, if this extension is not stable yet. Is it?

    --
    Stanislav Malyshev, Software Architect
    SugarCRM: http://www.sugarcrm.com/
    (408)454-6900 ext. 227
  • Pierre Joye at Aug 18, 2011 at 7:40 pm
    hi Stas!
    On Thu, Aug 18, 2011 at 7:27 PM, Stas Malyshev wrote:

    . not enabling it by default defeats the purpose of bundling it
    Actually, I don't think it does. It's one thing to tell people "put this
    option into php.ini" and quite another is "download it from PECL (?),
    compile it (?!) and install it". The first is within the limits of any
    literate person, the second is sysadmin territory.
    All major distros have it already, so it is the same. Now, if it is
    not enabled by default, then many admins won't install it, back to the
    same problem. It is very unlikely that someone with root or admin
    perms won't be able to install an extra extension from the distro
    repository or using the DLL on windows.
    . easier to support it while being only in pecl ('do you use the core
    version or latest pecl?")
    It is a concern, if this extension is not stable yet. Is it?
    It is stable. But the concern is not about its stability but the ease
    of support. Having it in core means than some users may use (for
    example) 5.4.2 version while we are at 5.4.6 or pecl has 3 new
    versions. Not a big deal, just easier.

    That being said, I'm not opposed to include it, I only don't see it as
    a must go :)

    Cheers,
  • Kingsquare.nl - Robin Speekenbrink at Aug 19, 2011 at 7:49 am
    Dear All,

    Again with my 2cents into the basket: I was hoping Ilia Alshanetsky
    would also support this idea since in his talks (i.e.
    http://ilia.ws/archives/241-IPC-Hidden-Features-of-PHP-Slides.html) he
    has been promoting the exact same use: switch the standard serializer
    to igBinary...

    Also the proposed include it -> read the data -> convert on write if in
    previous format (possibly an ini setting?) would provide full BC and
    would allow a painless upgrade... wouldnt it?

    --
    With kind regards,

    Robin Speekenbrink
    Kingsquare Information Services
    On donderdag 18 augustus 2011 21:39:56, Pierre Joye wrote:
    hi Stas!

    On Thu, Aug 18, 2011 at 7:27 PM, Stas Malyshevwrote:
    . not enabling it by default defeats the purpose of bundling it
    Actually, I don't think it does. It's one thing to tell people "put this
    option into php.ini" and quite another is "download it from PECL (?),
    compile it (?!) and install it". The first is wi
    thin the limits of any
    literate person, the second is sysadmin territory.
    All major distros have it already, so it is the same. Now, if it is
    not enabled by default, then many admins won't install it, back to the
    same problem. It is very unlikely that someone with root or admin
    perms won't be able to install an extra extension from the distro
    repository or using the DLL on windows.
    . easier to support it while being only in pecl ('do you use the core
    version or latest pecl?")
    It is a concern, if this extension is not stable yet. Is it?
    It is stable. But the concern is not about its stability but the ease
    of support. Having it in core means than some users may use (for
    example) 5.4.2 version while we are at 5.4.6 or pecl has 3 new
    versions. Not a big deal, just easier.

    That being said, I'm not opposed to include it, I only don't see it as
    a must go :)

    Cheers,
  • Sebastian Krebs at Aug 19, 2011 at 8:57 am
    Hi,

    One pitfall I see is, that two separate system can communicate using
    serialized data. They both must understand the same format and (in this
    case) must be configured equal.

    Regards,
    Sebastian

    2011/8/19 Kingsquare.nl - Robin Speekenbrink <robin@kingsquare.nl>
    Dear All,

    Again with my 2cents into the basket: I was hoping Ilia Alshanetsky would
    also support this idea since in his talks (i.e.
    http://ilia.ws/archives/241-**IPC-Hidden-Features-of-PHP-**Slides.html<http://ilia.ws/archives/241-IPC-Hidden-Features-of-PHP-Slides.html>)
    he has been promoting the exact same use: switch the standard serializer to
    igBinary...

    Also the proposed include it -> read the data -> convert on write if in
    previous format (possibly an ini setting?) would provide full BC and would
    allow a painless upgrade... wouldnt it?

    --
    With kind regards,

    Robin Speekenbrink
    Kingsquare Information Services
    On donderdag 18 augustus 2011 21:39:56, Pierre Joye wrote:

    hi Stas!

    On Thu, Aug 18, 2011 at 7:27 PM, Stas Malyshev<smalyshev@sugarcrm.**com<smalyshev@sugarcrm.com>>
    wrote:

    . not enabling it by default defeats the purpose of bundling it
    Actually, I don't think it does. It's one thing to tell people "put this
    option into php.ini" and quite another is "download it from PECL (?),
    compile it (?!) and install it". The first is wi
    thin the limits of any
    literate person, the second is sysadmin territory.
    All major distros have it already, so it is the same. Now, if it is
    not enabled by default, then many admins won't install it, back to the
    same problem. It is very unlikely that someone with root or admin
    perms won't be able to install an extra extension from the distro
    repository or using the DLL on windows.

    . easier to support it while being only in pecl ('do you use the core
    version or latest pecl?")
    It is a concern, if this extension is not stable yet. Is it?
    It is stable. But the concern is not about its stability but the ease
    of support. Having it in core means than some users may use (for
    example) 5.4.2 version while we are at 5.4.6 or pecl has 3 new
    versions. Not a big deal, just easier.

    That being said, I'm not opposed to include it, I only don't see it as
    a must go :)

    Cheers,
    --
    PHP Internals - PHP Runtime Development Mailing List
    To unsubscribe, visit: http://www.php.net/unsub.php
  • Jonathan Bond-Caron at Aug 19, 2011 at 11:15 am

    On Fri Aug 19 03:48 AM, Kingsquare.nl - Robin Speekenbrink wrote:

    Also the proposed include it -> read the data -> convert on write if
    in previous format (possibly an ini setting?) would provide full BC
    and would allow a painless upgrade... wouldnt it?
    100% against enabling it by default.

    - still a BC break since some apps in the wild assume serialize($foo) is ascii text (whether that's good or bad).
    - the default serialize() is readable, easy to debug, doesn't assume you can read binary. PHP is also about simplicity not performance.
    - igbinary_serialize_double() is not cross-platform so serializing/unserializing a 'float' won't always work
    http://www.dmh2000.com/cpp/dswap.shtml (2. use a 'union' type to overlay a byte array on top of the data)

    My preference would be to have:

    serialize_text() -- ascii text only (standard/var.c) but remove 'NULL' byte (working on a patch for this)
    serialize_binary() -- igbinary bundled into core...

    serialize() -- based on php.ini setting, serialize_text by default
  • Stefan Neufeind at Aug 19, 2011 at 11:24 am

    On 08/19/2011 01:15 PM, Jonathan Bond-Caron wrote:
    On Fri Aug 19 03:48 AM, Kingsquare.nl - Robin Speekenbrink wrote:

    Also the proposed include it -> read the data -> convert on write if
    in previous format (possibly an ini setting?) would provide full BC
    and would allow a painless upgrade... wouldnt it?
    100% against enabling it by default.

    - still a BC break since some apps in the wild assume serialize($foo) is ascii text (whether that's good or bad).
    - the default serialize() is readable, easy to debug, doesn't assume you can read binary. PHP is also about simplicity not performance.
    - igbinary_serialize_double() is not cross-platform so serializing/unserializing a 'float' won't always work
    http://www.dmh2000.com/cpp/dswap.shtml (2. use a 'union' type to overlay a byte array on top of the data)

    My preference would be to have:

    serialize_text() -- ascii text only (standard/var.c) but remove 'NULL' byte (working on a patch for this)
    serialize_binary() -- igbinary bundled into core...

    serialize() -- based on php.ini setting, serialize_text by default
    In general fine with me. But why not just give it a second (optional)
    parameter?


    Regards,
    Stefan
  • Todd Ruth at Sep 1, 2011 at 7:50 pm
    $x = null;
    get_class($x) returns get_class($this) as of 5.3.
    I don't recall seeing a discussion of this on the list and
    I can't find anything about this change in the archives.
    The documentation is currently self-contradictory...

    Under "Return Values":

    Returns the name of the class of which object is an instance. Returns
    FALSE if object is not an object.

    Under "Changelog":

    Since 5.3.0 NULL became the default value for object, so passing NULL
    to object now has the same result as not passing any value.

    (BTW, I don't see that listed in
    http://us.php.net/ChangeLog-5.php#5.3.0 )

    I guess there hasn't been a lot of screaming about it, so maybe
    it isn't a big deal, but it seems like a significant BC break to
    me. Maybe it's just because I've been beating my head against a
    wall for so long trying to figure out why my code was behaving so
    strangely. ;) I can't believe it's impossible to differentiate
    between receiving an argument and not receiving an argument.
    Would anyone really intentionally pass a null to this function
    to get the result of get_class($this)?

    Now that I know about this BC break, I'm OK and can avoid
    the issue, but for the sake of the next person to hit this,
    I felt I should post on the topic. I almost posted a bug
    directly, but the behavior is documented, so I thought this
    would be the correct place.

    - Todd
  • John Carter at Aug 19, 2011 at 9:15 am
    All,

    We're fairly heavy users of igbnary and we've found that the latest
    1.1.1 from pecl has issues with values-by-reference and therefore can't
    be used as a drop-in replacement for the standard serialiser as it
    stands.

    I suspect it's the same issue as this
    https://github.com/igbinary/igbinary/issues/6

    John.
    On Thu, 2011-08-18 at 12:12 +0100, Paul Dragoonis wrote:

    Hey guys,

    After a brief discussion with Pierre I'm taking this discussion to here.
    We have the igbinary pecl extension available [1], however not many of our
    userbase are familiar that it even exists.
    Here [2] is the performance boost we get from using igbinary with
    serialize() and unserialize().

    Here we get a great perf boost, and what's more excellent as we don't have
    to break BC to get there!

    This seems like a no-brainer and a good investment in the speed increase of
    PHP by default for the future users of the platform.

    [1] http://pecl.php.net/package/igbinary
    [2] http://www.php.net/~pierre/vcqa/apcigninary_perf.png

    Comments please.

    Regards,
    Paul Dragoonis.

    --
    John Carter
    Development Manager
    Identity Networks
  • Sebastian Bergmann at Aug 19, 2011 at 11:28 am

    Am 18.08.2011 13:12, schrieb Paul Dragoonis:
    Here we get a great perf boost, and what's more excellent as we don't have
    to break BC to get there!
    Are you sure it does not break BC? Does http://bit.ly/nIRxqz still work
    with IGBinary, for instance?

    --
    Sebastian Bergmann Co-Founder and Principal Consultant
    http://sebastian-bergmann.de/ http://thePHP.cc/
  • Pierre Joye at Aug 19, 2011 at 11:36 am

    On Fri, Aug 19, 2011 at 1:28 PM, Sebastian Bergmann wrote:
    Am 18.08.2011 13:12, schrieb Paul Dragoonis:
    Here we get a great perf boost, and what's more excellent as we don't have
    to break BC to get there!
    Are you sure it does not break BC? Does http://bit.ly/nIRxqz still work
    with IGBinary, for instance?
    That obviously won't work as long as igbinary is used. It is covered
    by the "interop" or BC break parts in my initial reply.

    Cheers,
  • Paul Dragoonis at Aug 19, 2011 at 11:42 am

    On Fri, Aug 19, 2011 at 12:36 PM, Pierre Joye wrote:
    On Fri, Aug 19, 2011 at 1:28 PM, Sebastian Bergmann wrote:
    Am 18.08.2011 13:12, schrieb Paul Dragoonis:
    Here we get a great perf boost, and what's more excellent as we don't
    have
    to break BC to get there!
    Are you sure it does not break BC? Does http://bit.ly/nIRxqz still work
    with IGBinary, for instance?
    That obviously won't work as long as igbinary is used. It is covered
    by the "interop" or BC break parts in my initial reply.
    So far, it looks like this "transparent" upgrade isn't that transparent
    anymore and will result in BC breaks. New approaches must be thought of.

    Like Pierre said, there will be new serialization engines in the future so
    it makes no sense to add more functions like serialize_binary() or whatnot.

    I think we need to keep our code base identical and maintain the
    serialize_handler via our php.ini configuration.

    Cheers,
    Paul.


    Cheers,
    --
    Pierre

    @pierrejoye | http://blog.thepimp.net | http://www.libgd.org

    --
    PHP Internals - PHP Runtime Development Mailing List
    To unsubscribe, visit: http://www.php.net/unsub.php
  • Ferenc Kovacs at Aug 19, 2011 at 12:04 pm

    On Fri, Aug 19, 2011 at 1:42 PM, Paul Dragoonis wrote:
    On Fri, Aug 19, 2011 at 12:36 PM, Pierre Joye wrote:

    On Fri, Aug 19, 2011 at 1:28 PM, Sebastian Bergmann <sebastian@php.net>
    wrote:
    Am 18.08.2011 13:12, schrieb Paul Dragoonis:
    Here we get a great perf boost, and what's more excellent as we don't
    have
    to break BC to get there!
    Are you sure it does not break BC? Does http://bit.ly/nIRxqz still work
    with IGBinary, for instance?
    That obviously won't work as long as igbinary is used. It is covered
    by the  "interop" or BC break parts in my initial reply.
    So far, it looks like this "transparent" upgrade isn't that transparent
    anymore and will result in BC breaks. New approaches must be thought of.

    Like Pierre said, there will be new serialization engines in the future so
    it makes no sense to add more functions like serialize_binary() or whatnot.

    I think we need to keep our code base identical and maintain the
    serialize_handler via our php.ini configuration.
    this could still cause problems, as pointed out before (when you
    change your handler, or when you pass serialized data between
    different php installs, etc.), so I really like what Kiall Mac Innes
    suggested: prefix the data with the serializer identifier.
    if it is missing, then it was serialized with the old method, if it
    is present, the appropriate method should be used, if the serializer
    ident is there but unknown, then exit with a fatal error telling the
    user that the custom handler is missing.

    the downside would be that if you want to serialize/unserialize the
    data outside of php, your implementation should take care of this
    prefix.
    just a wild idea, but maybe useful:
    instead of creating a prefix, we could serialize the passed data with
    the given(php, igbinary, etc.) handler, then wrap the whole stuff into
    an array which holds the name of the used handler and the serialized
    data, and serialize this array with the old(php) serialize method.
    this way the datablob would be always a valid serialized string, and
    would be easier to get the serialize method than with the prefixing.

    --
    Ferenc Kovács
    @Tyr43l - http://tyrael.hu
  • Arpad Ray at Aug 19, 2011 at 12:31 pm

    On Fri, Aug 19, 2011 at 1:04 PM, Ferenc Kovacs wrote:
    this could still cause problems, as pointed out before (when you
    change your handler, or when you pass serialized data between
    different php installs, etc.), so I really like what Kiall Mac Innes
    suggested: prefix the data with the serializer identifier.
    if it is missing,  then it was serialized with the old method, if it
    is present, the appropriate method should be used, if the serializer
    ident is there but unknown, then exit with a fatal error telling the
    user that the custom handler is missing.
    This came up when we discussing changing the default session
    serialization format a while ago. A prefix is a nice idea but care
    would have to be taken not to collide with any of the bytes which are
    valid for the current format to start with.

    Might be more of a curiosity than anything useful, but IIRC
    unserialize() currently ignores everything after the end of the data.
    the downside would be that if you want to serialize/unserialize the
    data outside of php, your implementation should take care of this
    prefix.
    just a wild idea, but maybe useful:
    instead of creating a prefix, we could serialize the passed data with
    the given(php, igbinary, etc.) handler, then wrap the whole stuff into
    an array which holds the name of the used handler and the serialized
    data, and serialize this array with the old(php) serialize method.
    this way the datablob would be always a valid serialized string, and
    would be easier to get the serialize method than with the prefixing.
    If my old app couldn't read some newly serialized string, I'd rather
    it failed hard than apparently succeed but have the wrong data.

    Regards,

    Arpad
  • Ferenc Kovacs at Aug 19, 2011 at 12:40 pm

    On Fri, Aug 19, 2011 at 2:31 PM, Arpad Ray wrote:
    On Fri, Aug 19, 2011 at 1:04 PM, Ferenc Kovacs wrote:
    this could still cause problems, as pointed out before (when you
    change your handler, or when you pass serialized data between
    different php installs, etc.), so I really like what Kiall Mac Innes
    suggested: prefix the data with the serializer identifier.
    if it is missing,  then it was serialized with the old method, if it
    is present, the appropriate method should be used, if the serializer
    ident is there but unknown, then exit with a fatal error telling the
    user that the custom handler is missing.
    This came up when we discussing changing the default session
    serialization format a while ago. A prefix is a nice idea but care
    would have to be taken not to collide with any of the bytes which are
    valid for the current format to start with.

    Might be more of a curiosity than anything useful, but IIRC
    unserialize() currently ignores everything after the end of the data.
    the downside would be that if you want to serialize/unserialize the
    data outside of php, your implementation should take care of this
    prefix.
    just a wild idea, but maybe useful:
    instead of creating a prefix, we could serialize the passed data with
    the given(php, igbinary, etc.) handler, then wrap the whole stuff into
    an array which holds the name of the used handler and the serialized
    data, and serialize this array with the old(php) serialize method.
    this way the datablob would be always a valid serialized string, and
    would be easier to get the serialize method than with the prefixing.
    If my old app couldn't read some newly serialized string, I'd rather
    it failed hard than apparently succeed but have the wrong data.
    http://php.net/unserialize
    In case the passed string is not unserializeable, FALSE is returned
    and E_NOTICE is issued.
    so if you unserialize from php, my suggestion would fail harder.
    when you unserialize it from another mean (parsing it on your own, or
    from a different language), you are right, it would produce a valid
    but different result what you would expect, but this could be
    documented, and I don't think that that many users do this, and maybe
    they are using json/bson/igbinary already.

    --
    Ferenc Kovács
    @Tyr43l - http://tyrael.hu
  • Arpad Ray at Aug 19, 2011 at 12:47 pm

    On Fri, Aug 19, 2011 at 1:40 PM, Ferenc Kovacs wrote:
    On Fri, Aug 19, 2011 at 2:31 PM, Arpad Ray wrote:
    On Fri, Aug 19, 2011 at 1:04 PM, Ferenc Kovacs wrote:
    the downside would be that if you want to serialize/unserialize the
    data outside of php, your implementation should take care of this
    prefix.
    just a wild idea, but maybe useful:
    instead of creating a prefix, we could serialize the passed data with
    the given(php, igbinary, etc.) handler, then wrap the whole stuff into
    an array which holds the name of the used handler and the serialized
    data, and serialize this array with the old(php) serialize method.
    this way the datablob would be always a valid serialized string, and
    would be easier to get the serialize method than with the prefixing.
    If my old app couldn't read some newly serialized string, I'd rather
    it failed hard than apparently succeed but have the wrong data.
    http://php.net/unserialize
    In case the passed string is not unserializeable, FALSE is returned
    and E_NOTICE is issued.
    so if you unserialize from php, my suggestion would fail harder.
    when you unserialize it from another mean (parsing it on your own, or
    from a different language), you are right, it would produce a valid
    but different result what you would expect, but this could be
    documented, and I don't think that that many users do this, and maybe
    they are using json/bson/igbinary already.
    I'm talking about unserialize() in an old (well, current) version of
    PHP - if it unserializes what is apparently a valid array there would
    be nothing to indicate it's actually a new version which it can't
    read.

    Regards,

    Arpad
  • Ferenc Kovacs at Aug 19, 2011 at 12:54 pm

    On Fri, Aug 19, 2011 at 2:47 PM, Arpad Ray wrote:
    On Fri, Aug 19, 2011 at 1:40 PM, Ferenc Kovacs wrote:
    On Fri, Aug 19, 2011 at 2:31 PM, Arpad Ray wrote:
    On Fri, Aug 19, 2011 at 1:04 PM, Ferenc Kovacs wrote:
    the downside would be that if you want to serialize/unserialize the
    data outside of php, your implementation should take care of this
    prefix.
    just a wild idea, but maybe useful:
    instead of creating a prefix, we could serialize the passed data with
    the given(php, igbinary, etc.) handler, then wrap the whole stuff into
    an array which holds the name of the used handler and the serialized
    data, and serialize this array with the old(php) serialize method.
    this way the datablob would be always a valid serialized string, and
    would be easier to get the serialize method than with the prefixing.
    If my old app couldn't read some newly serialized string, I'd rather
    it failed hard than apparently succeed but have the wrong data.
    http://php.net/unserialize
    In case the passed string is not unserializeable, FALSE is returned
    and E_NOTICE is issued.
    so if you unserialize from php, my suggestion would fail harder.
    when you unserialize it from another mean (parsing it on your own, or
    from a different language), you are right, it would produce a valid
    but different result what you would expect, but this could be
    documented, and I don't think that that many users do this, and maybe
    they are using json/bson/igbinary already.
    I'm talking about unserialize() in an old (well, current) version of
    PHP - if it unserializes what is apparently a valid array there would
    be nothing to indicate it's actually a new version which it can't
    read.
    I see, yeah, "old" clienst would still suffer from this, they would
    get an array from unserialize what they didn't expect, and maybe some
    data, what they cannot unpack (if it was packed with a method other
    than the current)
    the pro side is that it would be possible to handle this situation (if
    you cannot upgrade a php instance for example for some reason) as you
    could handle those situations (unserialize the data field to get the
    original data, or use igbinary from pecl to decompress it, etc.)
    I didn't feel too strong about this, just throwing ideas around.

    --
    Ferenc Kovács
    @Tyr43l - http://tyrael.hu
  • David Muir at Aug 21, 2011 at 11:49 pm

    On 08/19/2011 10:54 PM, Ferenc Kovacs wrote:
    On Fri, Aug 19, 2011 at 2:47 PM, Arpad Ray wrote:
    On Fri, Aug 19, 2011 at 1:40 PM, Ferenc Kovacs wrote:
    On Fri, Aug 19, 2011 at 2:31 PM, Arpad Ray wrote:
    On Fri, Aug 19, 2011 at 1:04 PM, Ferenc Kovacs wrote:
    the downside would be that if you want to serialize/unserialize the
    data outside of php, your implementation should take care of this
    prefix.
    just a wild idea, but maybe useful:
    instead of creating a prefix, we could serialize the passed data with
    the given(php, igbinary, etc.) handler, then wrap the whole stuff into
    an array which holds the name of the used handler and the serialized
    data, and serialize this array with the old(php) serialize method.
    this way the datablob would be always a valid serialized string, and
    would be easier to get the serialize method than with the prefixing.
    If my old app couldn't read some newly serialized string, I'd rather
    it failed hard than apparently succeed but have the wrong data.
    http://php.net/unserialize
    In case the passed string is not unserializeable, FALSE is returned
    and E_NOTICE is issued.
    so if you unserialize from php, my suggestion would fail harder.
    when you unserialize it from another mean (parsing it on your own, or
    from a different language), you are right, it would produce a valid
    but different result what you would expect, but this could be
    documented, and I don't think that that many users do this, and maybe
    they are using json/bson/igbinary already.
    I'm talking about unserialize() in an old (well, current) version of
    PHP - if it unserializes what is apparently a valid array there would
    be nothing to indicate it's actually a new version which it can't
    read.
    I see, yeah, "old" clienst would still suffer from this, they would
    get an array from unserialize what they didn't expect, and maybe some
    data, what they cannot unpack (if it was packed with a method other
    than the current)
    the pro side is that it would be possible to handle this situation (if
    you cannot upgrade a php instance for example for some reason) as you
    could handle those situations (unserialize the data field to get the
    original data, or use igbinary from pecl to decompress it, etc.)
    I didn't feel too strong about this, just throwing ideas around.
    Why not provide a userland fallback?

    eg: /IGBinary/unserialize()

    It could unserialize the string, albeit slowly, but still give you
    access to the data without needing the extension. At least that way you
    have something rather than nothing.

    David
  • Pierre Joye at Aug 22, 2011 at 12:07 am

    On Mon, Aug 22, 2011 at 1:49 AM, David Muir wrote:

    Why not provide a userland fallback?

    eg: /IGBinary/unserialize()

    It could unserialize the string, albeit slowly, but still give you
    access to the data without needing the extension. At least that way you
    have something rather than nothing.
    igbinary has functions for that already.
  • David Muir at Aug 23, 2011 at 1:11 am
    The gain is that those without the extension can still unserialize the
    data and read it. It solves the problem discussed where older versions
    of PHP without IGBinary built in would be locked out from being able to
    unserialise data.

    Didn't realise the reply was private. I did a reply-to-list, but
    Thunderbird picks up your address instead of internals.

    David
    On 23/08/11 09:52, Pierre Joye wrote:
    then the gain will be zero. By the way, no need to go private to reply :)
    On Tue, Aug 23, 2011 at 1:29 AM, David Muir wrote:
    That still requires the extension. I was referring to a userland
    implementation written in PHP, not C.

    Cheers,
    David
    On 22/08/11 17:21, Pierre Joye wrote:
    PHP_FE(igbinary_serialize, arginfo_igbinary_serialize)
    PHP_FE(igbinary_unserialize, arginfo_igbinary_unserialize)
    On Mon, Aug 22, 2011 at 2:18 AM, David Muir wrote:
    On 08/22/2011 10:07 AM, Pierre Joye wrote:
    On Mon, Aug 22, 2011 at 1:49 AM, David Muir wrote:

    Why not provide a userland fallback?

    eg: /IGBinary/unserialize()

    It could unserialize the string, albeit slowly, but still give you
    access to the data without needing the extension. At least that way you
    have something rather than nothing.
    igbinary has functions for that already.
    It does? I didn't see it in the docs, or in the git tree.

    Cheers,
    David
  • Jared Williams at Aug 19, 2011 at 1:01 pm

    -----Original Message-----
    From: Paul Dragoonis
    Sent: 18 August 2011 12:12
    To: PHP Internals List
    Subject: [PHP-DEV] Activation of IGBinary serialization
    extension in 5.4 by default

    Hey guys,

    After a brief discussion with Pierre I'm taking this
    discussion to here.
    We have the igbinary pecl extension available [1], however
    not many of our userbase are familiar that it even exists.
    Here [2] is the performance boost we get from using igbinary with
    serialize() and unserialize().

    Here we get a great perf boost, and what's more excellent as
    we don't have to break BC to get there!

    This seems like a no-brainer and a good investment in the
    speed increase of PHP by default for the future users of the platform.
    [1] http://pecl.php.net/package/igbinary
    [2] http://www.php.net/~pierre/vcqa/apcigninary_perf.png

    Comments please.

    Regards,
    Paul Dragoonis.
    I think MsgPack ( http://msgpack.org/ ) is a better option, as it is
    implementated in multiple languages.

    Jared
  • Lester Caine at Aug 19, 2011 at 8:45 pm

    Paul Dragoonis wrote:
    After a brief discussion with Pierre I'm taking this discussion to here.
    We have the igbinary pecl extension available [1], however not many of our
    userbase are familiar that it even exists.
    Here [2] is the performance boost we get from using igbinary with
    serialize() and unserialize().

    Here we get a great perf boost, and what's more excellent as we don't have
    to break BC to get there!

    This seems like a no-brainer and a good investment in the speed increase of
    PHP by default for the future users of the platform.

    [1]http://pecl.php.net/package/igbinary
    [2]http://www.php.net/~pierre/vcqa/apcigninary_perf.png

    Comments please.
    I find that the ability to quickly scan the session files when a client is
    having problems and identify their session is something that I use at least a
    couple of times a month on various remote sites. If the session data is not
    readable text then I can't do that. So anything that is done definitely needs to
    have an off switch as far as I am concerned. I only store a small amount of key
    data in the session - the rest is serialised into the clients database record,
    and I would prefer to use text blobs for that rather than having to manage
    binary data ...

    --
    Lester Caine - G8HFL
    -----------------------------
    Contact - http://lsces.co.uk/wiki/?page=contact
    L.S.Caine Electronic Services - http://lsces.co.uk
    EnquirySolve - http://enquirysolve.com/
    Model Engineers Digital Workshop - http://medw.co.uk//
    Firebird - http://www.firebirdsql.org/index.php

Related Discussions

People

Translate

site design / logo © 2022 Grokbase