Grokbase Groups Hive dev June 2014
FAQ
Douglas Moore created HIVE-7237:
-----------------------------------

              Summary: hive.exec.parallel=true w/ Hive 0.13/Tez causes application to linger forever
                  Key: HIVE-7237
                  URL: https://issues.apache.org/jira/browse/HIVE-7237
              Project: Hive
           Issue Type: Bug
           Components: Tez
     Affects Versions: 0.13.0
          Environment: HDP 2.1, Hive 0.13, SLES 11, 128GB data nodes, ORC SNAPPY
             Reporter: Douglas Moore


set hive.exec.parallel=true; will cause the Yarn application instance to linger
forever. set hive.exec.parallel=false, the application goes away as soon as hive query is complete. The underlying table is an ORC store_sales table compressed with SNAPPY.

{code}
hive.exec.parallel=true;
select * from store_sales where ss_ticket_number=5741230 and ss_item_sk=4825
{code}
The query will run under Tez and finish << 30 seconds.

After 30-40 of these jobs the cluster gets to a point where no jobs will finish.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Search Discussions

  • Navis (JIRA) at Jun 18, 2014 at 8:38 am
    [ https://issues.apache.org/jira/browse/HIVE-7237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Navis updated HIVE-7237:
    ------------------------

         Attachment: HIVE-7237.1.patch.txt
    hive.exec.parallel=true w/ Hive 0.13/Tez causes application to linger forever
    -----------------------------------------------------------------------------

    Key: HIVE-7237
    URL: https://issues.apache.org/jira/browse/HIVE-7237
    Project: Hive
    Issue Type: Bug
    Components: Tez
    Affects Versions: 0.13.0
    Environment: HDP 2.1, Hive 0.13, SLES 11, 128GB data nodes, ORC SNAPPY
    Reporter: Douglas Moore
    Attachments: HIVE-7237.1.patch.txt


    set hive.exec.parallel=true; will cause the Yarn application instance to linger
    forever. set hive.exec.parallel=false, the application goes away as soon as hive query is complete. The underlying table is an ORC store_sales table compressed with SNAPPY.
    {code}
    hive.exec.parallel=true;
    select * from store_sales where ss_ticket_number=5741230 and ss_item_sk=4825
    {code}
    The query will run under Tez and finish << 30 seconds.
    After 30-40 of these jobs the cluster gets to a point where no jobs will finish.


    --
    This message was sent by Atlassian JIRA
    (v6.2#6252)
  • Navis (JIRA) at Jun 18, 2014 at 8:38 am
    [ https://issues.apache.org/jira/browse/HIVE-7237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Navis updated HIVE-7237:
    ------------------------

         Assignee: Navis
           Status: Patch Available (was: Open)
    hive.exec.parallel=true w/ Hive 0.13/Tez causes application to linger forever
    -----------------------------------------------------------------------------

    Key: HIVE-7237
    URL: https://issues.apache.org/jira/browse/HIVE-7237
    Project: Hive
    Issue Type: Bug
    Components: Tez
    Affects Versions: 0.13.0
    Environment: HDP 2.1, Hive 0.13, SLES 11, 128GB data nodes, ORC SNAPPY
    Reporter: Douglas Moore
    Assignee: Navis
    Attachments: HIVE-7237.1.patch.txt


    set hive.exec.parallel=true; will cause the Yarn application instance to linger
    forever. set hive.exec.parallel=false, the application goes away as soon as hive query is complete. The underlying table is an ORC store_sales table compressed with SNAPPY.
    {code}
    hive.exec.parallel=true;
    select * from store_sales where ss_ticket_number=5741230 and ss_item_sk=4825
    {code}
    The query will run under Tez and finish << 30 seconds.
    After 30-40 of these jobs the cluster gets to a point where no jobs will finish.


    --
    This message was sent by Atlassian JIRA
    (v6.2#6252)
  • Hive QA (JIRA) at Jun 19, 2014 at 4:12 pm
    [ https://issues.apache.org/jira/browse/HIVE-7237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14037466#comment-14037466 ]

    Hive QA commented on HIVE-7237:
    -------------------------------



    {color:red}Overall{color}: -1 at least one tests failed

    Here are the results of testing the latest attachment:
    https://issues.apache.org/jira/secure/attachment/12651056/HIVE-7237.1.patch.txt

    {color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 5654 tests executed
    *Failed tests:*
    {noformat}
    org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_groupby1
    org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_root_dir_external_table
    org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_ctas
    {noformat}

    Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/516/testReport
    Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/516/console
    Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-Build-516/

    Messages:
    {noformat}
    Executing org.apache.hive.ptest.execution.PrepPhase
    Executing org.apache.hive.ptest.execution.ExecutionPhase
    Executing org.apache.hive.ptest.execution.ReportingPhase
    Tests exited with: TestsFailedException: 3 tests failed
    {noformat}

    This message is automatically generated.

    ATTACHMENT ID: 12651056
    hive.exec.parallel=true w/ Hive 0.13/Tez causes application to linger forever
    -----------------------------------------------------------------------------

    Key: HIVE-7237
    URL: https://issues.apache.org/jira/browse/HIVE-7237
    Project: Hive
    Issue Type: Bug
    Components: Tez
    Affects Versions: 0.13.0
    Environment: HDP 2.1, Hive 0.13, SLES 11, 128GB data nodes, ORC SNAPPY
    Reporter: Douglas Moore
    Assignee: Navis
    Attachments: HIVE-7237.1.patch.txt


    set hive.exec.parallel=true; will cause the Yarn application instance to linger
    forever. set hive.exec.parallel=false, the application goes away as soon as hive query is complete. The underlying table is an ORC store_sales table compressed with SNAPPY.
    {code}
    hive.exec.parallel=true;
    select * from store_sales where ss_ticket_number=5741230 and ss_item_sk=4825
    {code}
    The query will run under Tez and finish << 30 seconds.
    After 30-40 of these jobs the cluster gets to a point where no jobs will finish.


    --
    This message was sent by Atlassian JIRA
    (v6.2#6252)
  • Navis (JIRA) at Jun 23, 2014 at 1:19 am
    [ https://issues.apache.org/jira/browse/HIVE-7237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Navis updated HIVE-7237:
    ------------------------

         Attachment: HIVE-7237.2.patch.txt
    hive.exec.parallel=true w/ Hive 0.13/Tez causes application to linger forever
    -----------------------------------------------------------------------------

    Key: HIVE-7237
    URL: https://issues.apache.org/jira/browse/HIVE-7237
    Project: Hive
    Issue Type: Bug
    Components: Tez
    Affects Versions: 0.13.0
    Environment: HDP 2.1, Hive 0.13, SLES 11, 128GB data nodes, ORC SNAPPY
    Reporter: Douglas Moore
    Assignee: Navis
    Attachments: HIVE-7237.1.patch.txt, HIVE-7237.2.patch.txt


    set hive.exec.parallel=true; will cause the Yarn application instance to linger
    forever. set hive.exec.parallel=false, the application goes away as soon as hive query is complete. The underlying table is an ORC store_sales table compressed with SNAPPY.
    {code}
    hive.exec.parallel=true;
    select * from store_sales where ss_ticket_number=5741230 and ss_item_sk=4825
    {code}
    The query will run under Tez and finish << 30 seconds.
    After 30-40 of these jobs the cluster gets to a point where no jobs will finish.


    --
    This message was sent by Atlassian JIRA
    (v6.2#6252)
  • Hive QA (JIRA) at Jun 23, 2014 at 6:30 am
    [ https://issues.apache.org/jira/browse/HIVE-7237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14040459#comment-14040459 ]

    Hive QA commented on HIVE-7237:
    -------------------------------



    {color:red}Overall{color}: -1 at least one tests failed

    Here are the results of testing the latest attachment:
    https://issues.apache.org/jira/secure/attachment/12651909/HIVE-7237.2.patch.txt

    {color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 5669 tests executed
    *Failed tests:*
    {noformat}
    org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_dynpart_sort_optimization
    org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_root_dir_external_table
    org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_ctas
    {noformat}

    Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/556/testReport
    Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/556/console
    Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-Build-556/

    Messages:
    {noformat}
    Executing org.apache.hive.ptest.execution.PrepPhase
    Executing org.apache.hive.ptest.execution.ExecutionPhase
    Executing org.apache.hive.ptest.execution.ReportingPhase
    Tests exited with: TestsFailedException: 3 tests failed
    {noformat}

    This message is automatically generated.

    ATTACHMENT ID: 12651909
    hive.exec.parallel=true w/ Hive 0.13/Tez causes application to linger forever
    -----------------------------------------------------------------------------

    Key: HIVE-7237
    URL: https://issues.apache.org/jira/browse/HIVE-7237
    Project: Hive
    Issue Type: Bug
    Components: Tez
    Affects Versions: 0.13.0
    Environment: HDP 2.1, Hive 0.13, SLES 11, 128GB data nodes, ORC SNAPPY
    Reporter: Douglas Moore
    Assignee: Navis
    Attachments: HIVE-7237.1.patch.txt, HIVE-7237.2.patch.txt


    set hive.exec.parallel=true; will cause the Yarn application instance to linger
    forever. set hive.exec.parallel=false, the application goes away as soon as hive query is complete. The underlying table is an ORC store_sales table compressed with SNAPPY.
    {code}
    hive.exec.parallel=true;
    select * from store_sales where ss_ticket_number=5741230 and ss_item_sk=4825
    {code}
    The query will run under Tez and finish << 30 seconds.
    After 30-40 of these jobs the cluster gets to a point where no jobs will finish.


    --
    This message was sent by Atlassian JIRA
    (v6.2#6252)
  • Ashutosh Chauhan (JIRA) at Jun 23, 2014 at 9:36 am
    [ https://issues.apache.org/jira/browse/HIVE-7237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14040568#comment-14040568 ]

    Ashutosh Chauhan commented on HIVE-7237:
    ----------------------------------------

    +1
    hive.exec.parallel=true w/ Hive 0.13/Tez causes application to linger forever
    -----------------------------------------------------------------------------

    Key: HIVE-7237
    URL: https://issues.apache.org/jira/browse/HIVE-7237
    Project: Hive
    Issue Type: Bug
    Components: Tez
    Affects Versions: 0.13.0
    Environment: HDP 2.1, Hive 0.13, SLES 11, 128GB data nodes, ORC SNAPPY
    Reporter: Douglas Moore
    Assignee: Navis
    Attachments: HIVE-7237.1.patch.txt, HIVE-7237.2.patch.txt


    set hive.exec.parallel=true; will cause the Yarn application instance to linger
    forever. set hive.exec.parallel=false, the application goes away as soon as hive query is complete. The underlying table is an ORC store_sales table compressed with SNAPPY.
    {code}
    hive.exec.parallel=true;
    select * from store_sales where ss_ticket_number=5741230 and ss_item_sk=4825
    {code}
    The query will run under Tez and finish << 30 seconds.
    After 30-40 of these jobs the cluster gets to a point where no jobs will finish.


    --
    This message was sent by Atlassian JIRA
    (v6.2#6252)
  • Navis (JIRA) at Jun 24, 2014 at 12:58 am
    [ https://issues.apache.org/jira/browse/HIVE-7237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Navis updated HIVE-7237:
    ------------------------

            Resolution: Fixed
         Fix Version/s: 0.14.0
                Status: Resolved (was: Patch Available)

    Committed to trunk. Thanks Ashutosh, for the review.
    hive.exec.parallel=true w/ Hive 0.13/Tez causes application to linger forever
    -----------------------------------------------------------------------------

    Key: HIVE-7237
    URL: https://issues.apache.org/jira/browse/HIVE-7237
    Project: Hive
    Issue Type: Bug
    Components: Tez
    Affects Versions: 0.13.0
    Environment: HDP 2.1, Hive 0.13, SLES 11, 128GB data nodes, ORC SNAPPY
    Reporter: Douglas Moore
    Assignee: Navis
    Fix For: 0.14.0

    Attachments: HIVE-7237.1.patch.txt, HIVE-7237.2.patch.txt


    set hive.exec.parallel=true; will cause the Yarn application instance to linger
    forever. set hive.exec.parallel=false, the application goes away as soon as hive query is complete. The underlying table is an ORC store_sales table compressed with SNAPPY.
    {code}
    hive.exec.parallel=true;
    select * from store_sales where ss_ticket_number=5741230 and ss_item_sk=4825
    {code}
    The query will run under Tez and finish << 30 seconds.
    After 30-40 of these jobs the cluster gets to a point where no jobs will finish.


    --
    This message was sent by Atlassian JIRA
    (v6.2#6252)

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupdev @
categorieshive, hadoop
postedJun 16, '14 at 3:02p
activeJun 24, '14 at 12:58a
posts8
users1
websitehive.apache.org

1 user in discussion

Navis (JIRA): 8 posts

People

Translate

site design / logo © 2022 Grokbase