Grokbase Groups Pig dev August 2008
FAQ
When a pig script creates multiple MR jobs, if any job but the last fails, pig report success and returns no results.
---------------------------------------------------------------------------------------------------------------------

Key: PIG-392
URL: https://issues.apache.org/jira/browse/PIG-392
Project: Pig
Issue Type: Bug
Affects Versions: types_branch
Reporter: Alan Gates
Assignee: Alan Gates
Fix For: types_branch




--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Search Discussions

  • Alan Gates (JIRA) at Aug 22, 2008 at 3:29 pm
    [ https://issues.apache.org/jira/browse/PIG-392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Alan Gates updated PIG-392:
    ---------------------------

    Status: Patch Available (was: Open)
    When a pig script creates multiple MR jobs, if any job but the last fails, pig report success and returns no results.
    ---------------------------------------------------------------------------------------------------------------------

    Key: PIG-392
    URL: https://issues.apache.org/jira/browse/PIG-392
    Project: Pig
    Issue Type: Bug
    Affects Versions: types_branch
    Reporter: Alan Gates
    Assignee: Alan Gates
    Fix For: types_branch

    Attachments: PIG-392.patch

    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Alan Gates (JIRA) at Aug 22, 2008 at 3:29 pm
    [ https://issues.apache.org/jira/browse/PIG-392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Alan Gates updated PIG-392:
    ---------------------------

    Attachment: PIG-392.patch

    This patch changes the launcher classes to check for any failed jobs, instead of just checking that progress is 100% when hadoop declares all jobs finished.
    When a pig script creates multiple MR jobs, if any job but the last fails, pig report success and returns no results.
    ---------------------------------------------------------------------------------------------------------------------

    Key: PIG-392
    URL: https://issues.apache.org/jira/browse/PIG-392
    Project: Pig
    Issue Type: Bug
    Affects Versions: types_branch
    Reporter: Alan Gates
    Assignee: Alan Gates
    Fix For: types_branch

    Attachments: PIG-392.patch

    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Olga Natkovich (JIRA) at Aug 24, 2008 at 7:12 pm
    [ https://issues.apache.org/jira/browse/PIG-392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12625215#action_12625215 ]

    Olga Natkovich commented on PIG-392:
    ------------------------------------

    Alan, how do I validate the patch?
    When a pig script creates multiple MR jobs, if any job but the last fails, pig report success and returns no results.
    ---------------------------------------------------------------------------------------------------------------------

    Key: PIG-392
    URL: https://issues.apache.org/jira/browse/PIG-392
    Project: Pig
    Issue Type: Bug
    Affects Versions: types_branch
    Reporter: Alan Gates
    Assignee: Alan Gates
    Fix For: types_branch

    Attachments: PIG-392.patch

    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Shravan Matthur Narayanamurthy (JIRA) at Aug 25, 2008 at 10:58 am
    [ https://issues.apache.org/jira/browse/PIG-392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12625318#action_12625318 ]

    Shravan Matthur Narayanamurthy commented on PIG-392:
    ----------------------------------------------------

    But the calculateProgress does the same thing. It returns the progress of all the jobs as (sum of all successful jobs and running progress of running jobs). The launchers know how many jobs were launched. If JobControl works correctly, the completion check to see that the progress returned by calculateProgress equals num MR jobs submitted is right. Can you give me a sample script where it was failing?
    When a pig script creates multiple MR jobs, if any job but the last fails, pig report success and returns no results.
    ---------------------------------------------------------------------------------------------------------------------

    Key: PIG-392
    URL: https://issues.apache.org/jira/browse/PIG-392
    Project: Pig
    Issue Type: Bug
    Affects Versions: types_branch
    Reporter: Alan Gates
    Assignee: Alan Gates
    Fix For: types_branch

    Attachments: PIG-392.patch

    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Olga Natkovich (JIRA) at Aug 25, 2008 at 11:24 pm
    [ https://issues.apache.org/jira/browse/PIG-392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Olga Natkovich updated PIG-392:
    -------------------------------

    Resolution: Fixed
    Status: Resolved (was: Patch Available)

    patch committed. thanks, Alan
    When a pig script creates multiple MR jobs, if any job but the last fails, pig report success and returns no results.
    ---------------------------------------------------------------------------------------------------------------------

    Key: PIG-392
    URL: https://issues.apache.org/jira/browse/PIG-392
    Project: Pig
    Issue Type: Bug
    Affects Versions: types_branch
    Reporter: Alan Gates
    Assignee: Alan Gates
    Fix For: types_branch

    Attachments: PIG-392.patch

    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupdev @
categoriespig, hadoop
postedAug 21, '08 at 11:29p
activeAug 25, '08 at 11:24p
posts6
users1
websitepig.apache.org

1 user in discussion

Olga Natkovich (JIRA): 6 posts

People

Translate

site design / logo © 2022 Grokbase