Grokbase Groups Pig user January 2011
FAQ
I'm just curious how people usually interact with Maps? Do you write UDFs
that do interesting things with the maps and simply take advantage of the
fact that pig can deal with datatypes, or do you use java's functions on a
pig level? For things like constructing maps, doing a filter on whether a
given key or value is in a map, etc, just curious what people do. I looked
around and didn't see too much on it, if there's some documentation on use
I'd love to see it!

Search Discussions

  • Dmitriy Ryaboy at Jan 11, 2011 at 9:11 pm
    At Twitter, we tend to write UDFs because types are kind of broken for maps.
    Though a lot of stuff can be done in pig -- like if key->null and key not
    existing are the same thing for your, you can filter by mymap#'key' is not
    null.

    iirc some of the type brokenness is fixed in trunk.

    D
    On Tue, Jan 11, 2011 at 9:24 AM, Jonathan Coveney wrote:

    I'm just curious how people usually interact with Maps? Do you write UDFs
    that do interesting things with the maps and simply take advantage of the
    fact that pig can deal with datatypes, or do you use java's functions on a
    pig level? For things like constructing maps, doing a filter on whether a
    given key or value is in a map, etc, just curious what people do. I looked
    around and didn't see too much on it, if there's some documentation on use
    I'd love to see it!
  • Jonathan Coveney at Jan 11, 2011 at 9:41 pm
    Ah, cool. Just wanted to make sure I wasn't crazy before going UDF hogwild
    :)

    2011/1/11 Dmitriy Ryaboy <dvryaboy@gmail.com>
    At Twitter, we tend to write UDFs because types are kind of broken for
    maps.
    Though a lot of stuff can be done in pig -- like if key->null and key not
    existing are the same thing for your, you can filter by mymap#'key' is not
    null.

    iirc some of the type brokenness is fixed in trunk.

    D

    On Tue, Jan 11, 2011 at 9:24 AM, Jonathan Coveney <jcoveney@gmail.com
    wrote:
    I'm just curious how people usually interact with Maps? Do you write UDFs
    that do interesting things with the maps and simply take advantage of the
    fact that pig can deal with datatypes, or do you use java's functions on a
    pig level? For things like constructing maps, doing a filter on whether a
    given key or value is in a map, etc, just curious what people do. I looked
    around and didn't see too much on it, if there's some documentation on use
    I'd love to see it!

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupuser @
categoriespig, hadoop
postedJan 11, '11 at 5:24p
activeJan 11, '11 at 9:41p
posts3
users2
websitepig.apache.org

People

Translate

site design / logo © 2021 Grokbase