Grokbase Groups Pig dev November 2008
FAQ
Order of <key, value> pairs not preserved in MAP type.
------------------------------------------------------

Key: PIG-526
URL: https://issues.apache.org/jira/browse/PIG-526
Project: Pig
Issue Type: Bug
Components: data
Affects Versions: types_branch
Reporter: Hong Tang


PIG uses HashMap to deserialize the Pig MAP type which will not observe the order of <key, value> pairs.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Search Discussions

  • Hong Tang (JIRA) at Nov 13, 2008 at 7:41 pm
    [ https://issues.apache.org/jira/browse/PIG-526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12647370#action_12647370 ]

    Hong Tang commented on PIG-526:
    -------------------------------

    I suggest using LinkedHashMap instead.
    Order of <key, value> pairs not preserved in MAP type.
    ------------------------------------------------------

    Key: PIG-526
    URL: https://issues.apache.org/jira/browse/PIG-526
    Project: Pig
    Issue Type: Bug
    Components: data
    Affects Versions: types_branch
    Reporter: Hong Tang

    PIG uses HashMap to deserialize the Pig MAP type which will not observe the order of <key, value> pairs.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Alan Gates (JIRA) at Nov 13, 2008 at 8:05 pm
    [ https://issues.apache.org/jira/browse/PIG-526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12647375#action_12647375 ]

    Alan Gates commented on PIG-526:
    --------------------------------

    I don't know that pig promises any particular order for keys in a map. Do you believe we should be maintaining that order in memory, or is your concern that if we load and store the map it may be stored in a different order?
    Order of <key, value> pairs not preserved in MAP type.
    ------------------------------------------------------

    Key: PIG-526
    URL: https://issues.apache.org/jira/browse/PIG-526
    Project: Pig
    Issue Type: Bug
    Components: data
    Affects Versions: types_branch
    Reporter: Hong Tang

    PIG uses HashMap to deserialize the Pig MAP type which will not observe the order of <key, value> pairs.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Hong Tang (JIRA) at Nov 13, 2008 at 8:29 pm
    [ https://issues.apache.org/jira/browse/PIG-526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12647385#action_12647385 ]

    Hong Tang commented on PIG-526:
    -------------------------------

    Probably it is better for me to rephrase the issue as follows:

    Need <key, value> order control on PIG MAP type.

    For the lest, I would like to have the capability of keeping them sorted by key. Otherwise, it is more flexible to have PIG honor the ordering from the user-created Map object. The conceptual gap here is that: I can set a MAP datum using a SortedMap object when providing a tuple object; but after I write the tuple out and later read it back (through Writable interface), the iteration order of <key, value> pairs is completely changed.


    Order of <key, value> pairs not preserved in MAP type.
    ------------------------------------------------------

    Key: PIG-526
    URL: https://issues.apache.org/jira/browse/PIG-526
    Project: Pig
    Issue Type: Bug
    Components: data
    Affects Versions: types_branch
    Reporter: Hong Tang

    PIG uses HashMap to deserialize the Pig MAP type which will not observe the order of <key, value> pairs.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Olga Natkovich (JIRA) at Nov 13, 2008 at 10:10 pm
    [ https://issues.apache.org/jira/browse/PIG-526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12647424#action_12647424 ]

    Olga Natkovich commented on PIG-526:
    ------------------------------------

    My concern with forcing this is performance consequences of maintaing order. I don't think Pig promises to the users any particular order
    Order of <key, value> pairs not preserved in MAP type.
    ------------------------------------------------------

    Key: PIG-526
    URL: https://issues.apache.org/jira/browse/PIG-526
    Project: Pig
    Issue Type: Bug
    Components: data
    Affects Versions: types_branch
    Reporter: Hong Tang

    PIG uses HashMap to deserialize the Pig MAP type which will not observe the order of <key, value> pairs.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Hong Tang (JIRA) at Nov 13, 2008 at 10:28 pm
    [ https://issues.apache.org/jira/browse/PIG-526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12647432#action_12647432 ]

    Hong Tang commented on PIG-526:
    -------------------------------

    I understand your concern. But just as I rephrased, the issue here is that PIG allows no control from user side to choose which concrete Map class to use when deserilizing the tuples.

    Probably a good compromise is to allow user to specify which Map class should be used when performing Tuple deserialization.
    Order of <key, value> pairs not preserved in MAP type.
    ------------------------------------------------------

    Key: PIG-526
    URL: https://issues.apache.org/jira/browse/PIG-526
    Project: Pig
    Issue Type: Bug
    Components: data
    Affects Versions: types_branch
    Reporter: Hong Tang

    PIG uses HashMap to deserialize the Pig MAP type which will not observe the order of <key, value> pairs.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Olga Natkovich (JIRA) at Nov 14, 2008 at 12:03 am
    [ https://issues.apache.org/jira/browse/PIG-526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12647474#action_12647474 ]

    Olga Natkovich commented on PIG-526:
    ------------------------------------

    As of now Pig does not guarantee any order on the keys and we have not seen users complaining about it. So I am not sure that we need to do anything. Also, at the moment (until we move to 3.0 hopefully not soon.) we can't e breaking any existing interfaces.
    Order of <key, value> pairs not preserved in MAP type.
    ------------------------------------------------------

    Key: PIG-526
    URL: https://issues.apache.org/jira/browse/PIG-526
    Project: Pig
    Issue Type: Bug
    Components: data
    Affects Versions: types_branch
    Reporter: Hong Tang

    PIG uses HashMap to deserialize the Pig MAP type which will not observe the order of <key, value> pairs.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Hong Tang (JIRA) at Jul 31, 2009 at 2:06 am
    [ https://issues.apache.org/jira/browse/PIG-526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Hong Tang resolved PIG-526.
    ---------------------------

    Resolution: Won't Fix
    Order of <key, value> pairs not preserved in MAP type.
    ------------------------------------------------------

    Key: PIG-526
    URL: https://issues.apache.org/jira/browse/PIG-526
    Project: Pig
    Issue Type: Bug
    Components: data
    Affects Versions: 0.2.0
    Reporter: Hong Tang

    PIG uses HashMap to deserialize the Pig MAP type which will not observe the order of <key, value> pairs.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupdev @
categoriespig, hadoop
postedNov 13, '08 at 7:41p
activeJul 31, '09 at 2:06a
posts8
users1
websitepig.apache.org

1 user in discussion

Hong Tang (JIRA): 8 posts

People

Translate

site design / logo © 2022 Grokbase