FAQ

[Thrift-user] union support

Philip Winston
Apr 20, 2011 at 4:25 pm
We are just starting to design a Thrift interface. I read about "thrift
union" workaround and then subsequent addition of real union support in the
trunk:

http://blog.rapleaf.com/dev/2009/06/03/thrift-union-pattern/
http://blog.rapleaf.com/dev/2009/08/18/thrift-unions-part-ii-or-how-i-reduced-memory-usage-by-95/

I'm wondering what the status of this is, because I don't see it mentioned
in the tutorial or thrift language specification. Is it in the trunk? Do
people use this, does it work well for you?

I'm also interested in understanding the workaround (without real union
support). How "bad" is the workaround in terms of memory usage? Thrift has
"sparse structs" so I imagine you never had to pay the cost of the full
compound structure and all substructures. Instead does the real union
support basically just save you the cost of the null for each "empty"
struct? Is the "struct_type" discriminating field strictly needed, can you
just query for which component struct is non-empty? I guess having the type
field is just faster?

Thanks.

-Philip
reply

Search Discussions

2 responses

  • Bryan Duxbury at Apr 20, 2011 at 5:13 pm
    There is full-fledged support for Unions in the IDL and the Java and Ruby
    libraries. Other languages will just generate a struct with the same set of
    fields, so they are backwards compatible.

    The "workaround" version can be pretty bad. You are correct that you don't
    have to pay the price of any sub-structure, but having many nulls can be
    pretty memory-intensive. In Java, for instance, you have to spend 4/8 bytes
    per null reference, and if you have a lot of fields, it can add up fast.

    You don't strictly need a discriminating field, but in practice, it'll
    really help you out. There's no direct way of querying which field is set -
    Unions have support for that, but it doesn't make sense in a regular struct.
    You could iterate over all the fields and check which one is set, but I
    could imagine that being slow.
    On Wed, Apr 20, 2011 at 9:24 AM, Philip Winston wrote:

    We are just starting to design a Thrift interface. I read about "thrift
    union" workaround and then subsequent addition of real union support in the
    trunk:

    http://blog.rapleaf.com/dev/2009/06/03/thrift-union-pattern/

    http://blog.rapleaf.com/dev/2009/08/18/thrift-unions-part-ii-or-how-i-reduced-memory-usage-by-95/

    I'm wondering what the status of this is, because I don't see it mentioned
    in the tutorial or thrift language specification. Is it in the trunk? Do
    people use this, does it work well for you?

    I'm also interested in understanding the workaround (without real union
    support). How "bad" is the workaround in terms of memory usage? Thrift
    has
    "sparse structs" so I imagine you never had to pay the cost of the full
    compound structure and all substructures. Instead does the real union
    support basically just save you the cost of the null for each "empty"
    struct? Is the "struct_type" discriminating field strictly needed, can you
    just query for which component struct is non-empty? I guess having the
    type
    field is just faster?

    Thanks.

    -Philip
  • Richard Salz at Apr 20, 2011 at 6:13 pm

    You don't strictly need a discriminating field, but in practice, it'll
    really help you out.
    Yeah, I'll say. :) I would be in favor of making it mandatory for true
    union support.

    /r$

Related Discussions

Discussion Navigation
viewthread | post