Grokbase Groups Pig dev August 2010
FAQ
native mapreduce operator MR job does not follow same failure handling logic as other pig MR jobs
-------------------------------------------------------------------------------------------------

Key: PIG-1570
URL: https://issues.apache.org/jira/browse/PIG-1570
Project: Pig
Issue Type: Bug
Reporter: Thejas M Nair
Fix For: 0.8.0


The code path for handling failure in MR job corresponding to native MR is different and does not have the same behavior.
For example, even if the MR job for mapreduce operator fails, the number of jobs that failed is being reported as 0 in PigStats log.


--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Search Discussions

  • Thejas M Nair (JIRA) at Aug 26, 2010 at 4:32 pm
    [ https://issues.apache.org/jira/browse/PIG-1570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12902919#action_12902919 ]

    Thejas M Nair commented on PIG-1570:
    ------------------------------------

    Another thing to investigate (somewhat related) - there seems to be a problem when PigServer is used to execute query having native mr operator - i was unable to run the tests in local mode . But i am able to run query in local mode from commandline.

    native mapreduce operator MR job does not follow same failure handling logic as other pig MR jobs
    -------------------------------------------------------------------------------------------------

    Key: PIG-1570
    URL: https://issues.apache.org/jira/browse/PIG-1570
    Project: Pig
    Issue Type: Bug
    Reporter: Thejas M Nair
    Fix For: 0.8.0


    The code path for handling failure in MR job corresponding to native MR is different and does not have the same behavior.
    For example, even if the MR job for mapreduce operator fails, the number of jobs that failed is being reported as 0 in PigStats log.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Olga Natkovich (JIRA) at Aug 27, 2010 at 8:31 pm
    [ https://issues.apache.org/jira/browse/PIG-1570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Olga Natkovich reassigned PIG-1570:
    -----------------------------------

    Assignee: Thejas M Nair
    native mapreduce operator MR job does not follow same failure handling logic as other pig MR jobs
    -------------------------------------------------------------------------------------------------

    Key: PIG-1570
    URL: https://issues.apache.org/jira/browse/PIG-1570
    Project: Pig
    Issue Type: Bug
    Reporter: Thejas M Nair
    Assignee: Thejas M Nair
    Fix For: 0.8.0


    The code path for handling failure in MR job corresponding to native MR is different and does not have the same behavior.
    For example, even if the MR job for mapreduce operator fails, the number of jobs that failed is being reported as 0 in PigStats log.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Thejas M Nair (JIRA) at Aug 30, 2010 at 5:24 pm
    [ https://issues.apache.org/jira/browse/PIG-1570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12904258#action_12904258 ]

    Thejas M Nair commented on PIG-1570:
    ------------------------------------

    Regarding
    bq. Another thing to investigate (somewhat related) - there seems to be a problem when PigServer is used to execute query having native mr operator - i was unable to run the tests in local mode . But i am able to run query in local mode from commandline.

    The problem was that in test setup, the MiniCluster hadoop-site.xml (~/pigtest/conf/hadoop-site.xml) is in classpath. The WordCount.jar would end up trying to run the MR job using minicluster and fail, if rest of the test is using local mode.

    native mapreduce operator MR job does not follow same failure handling logic as other pig MR jobs
    -------------------------------------------------------------------------------------------------

    Key: PIG-1570
    URL: https://issues.apache.org/jira/browse/PIG-1570
    Project: Pig
    Issue Type: Bug
    Reporter: Thejas M Nair
    Assignee: Thejas M Nair
    Fix For: 0.8.0


    The code path for handling failure in MR job corresponding to native MR is different and does not have the same behavior.
    For example, even if the MR job for mapreduce operator fails, the number of jobs that failed is being reported as 0 in PigStats log.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Thejas M Nair (JIRA) at Aug 30, 2010 at 5:26 pm
    [ https://issues.apache.org/jira/browse/PIG-1570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Thejas M Nair updated PIG-1570:
    -------------------------------

    Attachment: PIG-1570.1.patch

    Patch passed test-patch and core tests. Patch is ready for review.
    [exec] +1 overall.
    [exec]
    [exec] +1 @author. The patch does not contain any @author tags.
    [exec]
    [exec] +1 tests included. The patch appears to include 5 new or modified tests.
    [exec]
    [exec] +1 javadoc. The javadoc tool did not generate any warning messages.
    [exec]
    [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings.
    [exec]
    [exec] +1 findbugs. The patch does not introduce any new Findbugs warnings.
    [exec]
    [exec] +1 release audit. The applied patch does not increase the total number of release audit warnings.
    [exec]

    native mapreduce operator MR job does not follow same failure handling logic as other pig MR jobs
    -------------------------------------------------------------------------------------------------

    Key: PIG-1570
    URL: https://issues.apache.org/jira/browse/PIG-1570
    Project: Pig
    Issue Type: Bug
    Reporter: Thejas M Nair
    Assignee: Thejas M Nair
    Fix For: 0.8.0

    Attachments: PIG-1570.1.patch


    The code path for handling failure in MR job corresponding to native MR is different and does not have the same behavior.
    For example, even if the MR job for mapreduce operator fails, the number of jobs that failed is being reported as 0 in PigStats log.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Thejas M Nair (JIRA) at Aug 30, 2010 at 5:30 pm
    [ https://issues.apache.org/jira/browse/PIG-1570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12904263#action_12904263 ]

    Thejas M Nair commented on PIG-1570:
    ------------------------------------

    The code path that is followed in case of the native MR job is still different because the jar is a black box, and pig just calls the main function, pig doesn't even know if it is a MR job that is actually being run.
    This fixes the pig stats reporting (log messages) for failed native MR job and also the feature list in the native MR job.

    native mapreduce operator MR job does not follow same failure handling logic as other pig MR jobs
    -------------------------------------------------------------------------------------------------

    Key: PIG-1570
    URL: https://issues.apache.org/jira/browse/PIG-1570
    Project: Pig
    Issue Type: Bug
    Reporter: Thejas M Nair
    Assignee: Thejas M Nair
    Fix For: 0.8.0

    Attachments: PIG-1570.1.patch


    The code path for handling failure in MR job corresponding to native MR is different and does not have the same behavior.
    For example, even if the MR job for mapreduce operator fails, the number of jobs that failed is being reported as 0 in PigStats log.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Richard Ding (JIRA) at Aug 30, 2010 at 8:24 pm
    [ https://issues.apache.org/jira/browse/PIG-1570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12904321#action_12904321 ]

    Richard Ding commented on PIG-1570:
    -----------------------------------

    +1.
    native mapreduce operator MR job does not follow same failure handling logic as other pig MR jobs
    -------------------------------------------------------------------------------------------------

    Key: PIG-1570
    URL: https://issues.apache.org/jira/browse/PIG-1570
    Project: Pig
    Issue Type: Bug
    Reporter: Thejas M Nair
    Assignee: Thejas M Nair
    Fix For: 0.8.0

    Attachments: PIG-1570.1.patch


    The code path for handling failure in MR job corresponding to native MR is different and does not have the same behavior.
    For example, even if the MR job for mapreduce operator fails, the number of jobs that failed is being reported as 0 in PigStats log.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Thejas M Nair (JIRA) at Aug 31, 2010 at 12:10 am
    [ https://issues.apache.org/jira/browse/PIG-1570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Thejas M Nair resolved PIG-1570.
    --------------------------------

    Hadoop Flags: [Reviewed]
    Resolution: Fixed

    Patch committed to trunk.
    native mapreduce operator MR job does not follow same failure handling logic as other pig MR jobs
    -------------------------------------------------------------------------------------------------

    Key: PIG-1570
    URL: https://issues.apache.org/jira/browse/PIG-1570
    Project: Pig
    Issue Type: Bug
    Reporter: Thejas M Nair
    Assignee: Thejas M Nair
    Fix For: 0.8.0

    Attachments: PIG-1570.1.patch


    The code path for handling failure in MR job corresponding to native MR is different and does not have the same behavior.
    For example, even if the MR job for mapreduce operator fails, the number of jobs that failed is being reported as 0 in PigStats log.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupdev @
categoriespig, hadoop
postedAug 26, '10 at 3:51p
activeAug 31, '10 at 12:10a
posts8
users1
websitepig.apache.org

1 user in discussion

Thejas M Nair (JIRA): 8 posts

People

Translate

site design / logo © 2022 Grokbase