Grokbase Groups Hive dev October 2014
FAQ
[ https://issues.apache.org/jira/browse/HIVE-8225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14154497#comment-14154497 ]

Hive QA commented on HIVE-8225:
-------------------------------



{color:red}Overall{color}: -1 at least one tests failed

Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12672197/HIVE-8225.1.patch

{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 6379 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_cbo_correctness
{noformat}

Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/1066/testReport
Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/1066/console
Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-1066/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12672197
CBO trunk merge: union11 test fails due to incorrect plan
---------------------------------------------------------

Key: HIVE-8225
URL: https://issues.apache.org/jira/browse/HIVE-8225
Project: Hive
Issue Type: Bug
Reporter: Sergey Shelukhin
Assignee: Sergey Shelukhin
Attachments: HIVE-8225.1.patch, HIVE-8225.inprogress.patch, HIVE-8225.inprogress.patch, HIVE-8225.patch


The result changes to as if the union didn't have count() inside. The issue can be fixed by using srcunion.value outside the subquery in count (replace count(1) with count(srcunion.value)). Otherwise, it looks like count(1) node from union-ed queries is not present in AST at all, which might cause this result.
-Interestingly, adding group by to each query in a union produces completely weird result (count(1) is 309 for each key, whereas it should be 1 and the "logical" incorrect value if internal count is lost is 500)- Nm, that groups by table column called key, which is weird but is what Hive does


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Search Discussions

  • Vikram Dixit K (JIRA) at Oct 1, 2014 at 4:29 pm
    [ https://issues.apache.org/jira/browse/HIVE-8225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Vikram Dixit K updated HIVE-8225:
    ---------------------------------
         Priority: Critical (was: Major)
    CBO trunk merge: union11 test fails due to incorrect plan
    ---------------------------------------------------------

    Key: HIVE-8225
    URL: https://issues.apache.org/jira/browse/HIVE-8225
    Project: Hive
    Issue Type: Bug
    Reporter: Sergey Shelukhin
    Assignee: Sergey Shelukhin
    Priority: Critical
    Fix For: 0.14.0

    Attachments: HIVE-8225.1.patch, HIVE-8225.inprogress.patch, HIVE-8225.inprogress.patch, HIVE-8225.patch


    The result changes to as if the union didn't have count() inside. The issue can be fixed by using srcunion.value outside the subquery in count (replace count(1) with count(srcunion.value)). Otherwise, it looks like count(1) node from union-ed queries is not present in AST at all, which might cause this result.
    -Interestingly, adding group by to each query in a union produces completely weird result (count(1) is 309 for each key, whereas it should be 1 and the "logical" incorrect value if internal count is lost is 500)- Nm, that groups by table column called key, which is weird but is what Hive does


    --
    This message was sent by Atlassian JIRA
    (v6.3.4#6332)
  • Vikram Dixit K (JIRA) at Oct 1, 2014 at 4:29 pm
    [ https://issues.apache.org/jira/browse/HIVE-8225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Vikram Dixit K updated HIVE-8225:
    ---------------------------------
         Fix Version/s: 0.14.0
    CBO trunk merge: union11 test fails due to incorrect plan
    ---------------------------------------------------------

    Key: HIVE-8225
    URL: https://issues.apache.org/jira/browse/HIVE-8225
    Project: Hive
    Issue Type: Bug
    Reporter: Sergey Shelukhin
    Assignee: Sergey Shelukhin
    Priority: Critical
    Fix For: 0.14.0

    Attachments: HIVE-8225.1.patch, HIVE-8225.inprogress.patch, HIVE-8225.inprogress.patch, HIVE-8225.patch


    The result changes to as if the union didn't have count() inside. The issue can be fixed by using srcunion.value outside the subquery in count (replace count(1) with count(srcunion.value)). Otherwise, it looks like count(1) node from union-ed queries is not present in AST at all, which might cause this result.
    -Interestingly, adding group by to each query in a union produces completely weird result (count(1) is 309 for each key, whereas it should be 1 and the "logical" incorrect value if internal count is lost is 500)- Nm, that groups by table column called key, which is weird but is what Hive does


    --
    This message was sent by Atlassian JIRA
    (v6.3.4#6332)
  • Pengcheng Xiong (JIRA) at Oct 1, 2014 at 5:16 pm
    [ https://issues.apache.org/jira/browse/HIVE-8225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14155137#comment-14155137 ]

    Pengcheng Xiong commented on HIVE-8225:
    ---------------------------------------

    patch already includes the update of cbo-correctness.q and q.out. how come it still use outdated q.out file?
    CBO trunk merge: union11 test fails due to incorrect plan
    ---------------------------------------------------------

    Key: HIVE-8225
    URL: https://issues.apache.org/jira/browse/HIVE-8225
    Project: Hive
    Issue Type: Bug
    Reporter: Sergey Shelukhin
    Assignee: Sergey Shelukhin
    Priority: Critical
    Fix For: 0.14.0

    Attachments: HIVE-8225.1.patch, HIVE-8225.inprogress.patch, HIVE-8225.inprogress.patch, HIVE-8225.patch


    The result changes to as if the union didn't have count() inside. The issue can be fixed by using srcunion.value outside the subquery in count (replace count(1) with count(srcunion.value)). Otherwise, it looks like count(1) node from union-ed queries is not present in AST at all, which might cause this result.
    -Interestingly, adding group by to each query in a union produces completely weird result (count(1) is 309 for each key, whereas it should be 1 and the "logical" incorrect value if internal count is lost is 500)- Nm, that groups by table column called key, which is weird but is what Hive does


    --
    This message was sent by Atlassian JIRA
    (v6.3.4#6332)
  • Pengcheng Xiong (JIRA) at Oct 1, 2014 at 5:36 pm
    [ https://issues.apache.org/jira/browse/HIVE-8225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Pengcheng Xiong updated HIVE-8225:
    ----------------------------------
         Status: Open (was: Patch Available)
    CBO trunk merge: union11 test fails due to incorrect plan
    ---------------------------------------------------------

    Key: HIVE-8225
    URL: https://issues.apache.org/jira/browse/HIVE-8225
    Project: Hive
    Issue Type: Bug
    Reporter: Sergey Shelukhin
    Assignee: Sergey Shelukhin
    Priority: Critical
    Fix For: 0.14.0

    Attachments: HIVE-8225.1.patch, HIVE-8225.inprogress.patch, HIVE-8225.inprogress.patch, HIVE-8225.patch


    The result changes to as if the union didn't have count() inside. The issue can be fixed by using srcunion.value outside the subquery in count (replace count(1) with count(srcunion.value)). Otherwise, it looks like count(1) node from union-ed queries is not present in AST at all, which might cause this result.
    -Interestingly, adding group by to each query in a union produces completely weird result (count(1) is 309 for each key, whereas it should be 1 and the "logical" incorrect value if internal count is lost is 500)- Nm, that groups by table column called key, which is weird but is what Hive does


    --
    This message was sent by Atlassian JIRA
    (v6.3.4#6332)
  • Pengcheng Xiong (JIRA) at Oct 1, 2014 at 5:37 pm
    [ https://issues.apache.org/jira/browse/HIVE-8225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Pengcheng Xiong updated HIVE-8225:
    ----------------------------------
         Attachment: HIVE-8225.2.patch

    update q.out for tez
    CBO trunk merge: union11 test fails due to incorrect plan
    ---------------------------------------------------------

    Key: HIVE-8225
    URL: https://issues.apache.org/jira/browse/HIVE-8225
    Project: Hive
    Issue Type: Bug
    Reporter: Sergey Shelukhin
    Assignee: Sergey Shelukhin
    Priority: Critical
    Fix For: 0.14.0

    Attachments: HIVE-8225.1.patch, HIVE-8225.2.patch, HIVE-8225.inprogress.patch, HIVE-8225.inprogress.patch, HIVE-8225.patch


    The result changes to as if the union didn't have count() inside. The issue can be fixed by using srcunion.value outside the subquery in count (replace count(1) with count(srcunion.value)). Otherwise, it looks like count(1) node from union-ed queries is not present in AST at all, which might cause this result.
    -Interestingly, adding group by to each query in a union produces completely weird result (count(1) is 309 for each key, whereas it should be 1 and the "logical" incorrect value if internal count is lost is 500)- Nm, that groups by table column called key, which is weird but is what Hive does


    --
    This message was sent by Atlassian JIRA
    (v6.3.4#6332)
  • Pengcheng Xiong (JIRA) at Oct 1, 2014 at 5:37 pm
    [ https://issues.apache.org/jira/browse/HIVE-8225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Pengcheng Xiong updated HIVE-8225:
    ----------------------------------
         Status: Patch Available (was: Open)
    CBO trunk merge: union11 test fails due to incorrect plan
    ---------------------------------------------------------

    Key: HIVE-8225
    URL: https://issues.apache.org/jira/browse/HIVE-8225
    Project: Hive
    Issue Type: Bug
    Reporter: Sergey Shelukhin
    Assignee: Sergey Shelukhin
    Priority: Critical
    Fix For: 0.14.0

    Attachments: HIVE-8225.1.patch, HIVE-8225.2.patch, HIVE-8225.inprogress.patch, HIVE-8225.inprogress.patch, HIVE-8225.patch


    The result changes to as if the union didn't have count() inside. The issue can be fixed by using srcunion.value outside the subquery in count (replace count(1) with count(srcunion.value)). Otherwise, it looks like count(1) node from union-ed queries is not present in AST at all, which might cause this result.
    -Interestingly, adding group by to each query in a union produces completely weird result (count(1) is 309 for each key, whereas it should be 1 and the "logical" incorrect value if internal count is lost is 500)- Nm, that groups by table column called key, which is weird but is what Hive does


    --
    This message was sent by Atlassian JIRA
    (v6.3.4#6332)
  • Pengcheng Xiong (JIRA) at Oct 1, 2014 at 8:47 pm
    [ https://issues.apache.org/jira/browse/HIVE-8225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Pengcheng Xiong updated HIVE-8225:
    ----------------------------------
         Status: Open (was: Patch Available)
    CBO trunk merge: union11 test fails due to incorrect plan
    ---------------------------------------------------------

    Key: HIVE-8225
    URL: https://issues.apache.org/jira/browse/HIVE-8225
    Project: Hive
    Issue Type: Bug
    Reporter: Sergey Shelukhin
    Assignee: Sergey Shelukhin
    Priority: Critical
    Fix For: 0.14.0

    Attachments: HIVE-8225.1.patch, HIVE-8225.2.patch, HIVE-8225.inprogress.patch, HIVE-8225.inprogress.patch, HIVE-8225.patch


    The result changes to as if the union didn't have count() inside. The issue can be fixed by using srcunion.value outside the subquery in count (replace count(1) with count(srcunion.value)). Otherwise, it looks like count(1) node from union-ed queries is not present in AST at all, which might cause this result.
    -Interestingly, adding group by to each query in a union produces completely weird result (count(1) is 309 for each key, whereas it should be 1 and the "logical" incorrect value if internal count is lost is 500)- Nm, that groups by table column called key, which is weird but is what Hive does


    --
    This message was sent by Atlassian JIRA
    (v6.3.4#6332)
  • Pengcheng Xiong (JIRA) at Oct 1, 2014 at 8:48 pm
    [ https://issues.apache.org/jira/browse/HIVE-8225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Pengcheng Xiong updated HIVE-8225:
    ----------------------------------
         Attachment: HIVE-8225.3.patch

    rebase to trunk after John rename the derivedtable java file
    CBO trunk merge: union11 test fails due to incorrect plan
    ---------------------------------------------------------

    Key: HIVE-8225
    URL: https://issues.apache.org/jira/browse/HIVE-8225
    Project: Hive
    Issue Type: Bug
    Reporter: Sergey Shelukhin
    Assignee: Sergey Shelukhin
    Priority: Critical
    Fix For: 0.14.0

    Attachments: HIVE-8225.1.patch, HIVE-8225.2.patch, HIVE-8225.3.patch, HIVE-8225.inprogress.patch, HIVE-8225.inprogress.patch, HIVE-8225.patch


    The result changes to as if the union didn't have count() inside. The issue can be fixed by using srcunion.value outside the subquery in count (replace count(1) with count(srcunion.value)). Otherwise, it looks like count(1) node from union-ed queries is not present in AST at all, which might cause this result.
    -Interestingly, adding group by to each query in a union produces completely weird result (count(1) is 309 for each key, whereas it should be 1 and the "logical" incorrect value if internal count is lost is 500)- Nm, that groups by table column called key, which is weird but is what Hive does


    --
    This message was sent by Atlassian JIRA
    (v6.3.4#6332)
  • Pengcheng Xiong (JIRA) at Oct 1, 2014 at 8:48 pm
    [ https://issues.apache.org/jira/browse/HIVE-8225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Pengcheng Xiong updated HIVE-8225:
    ----------------------------------
         Status: Patch Available (was: Open)
    CBO trunk merge: union11 test fails due to incorrect plan
    ---------------------------------------------------------

    Key: HIVE-8225
    URL: https://issues.apache.org/jira/browse/HIVE-8225
    Project: Hive
    Issue Type: Bug
    Reporter: Sergey Shelukhin
    Assignee: Sergey Shelukhin
    Priority: Critical
    Fix For: 0.14.0

    Attachments: HIVE-8225.1.patch, HIVE-8225.2.patch, HIVE-8225.3.patch, HIVE-8225.inprogress.patch, HIVE-8225.inprogress.patch, HIVE-8225.patch


    The result changes to as if the union didn't have count() inside. The issue can be fixed by using srcunion.value outside the subquery in count (replace count(1) with count(srcunion.value)). Otherwise, it looks like count(1) node from union-ed queries is not present in AST at all, which might cause this result.
    -Interestingly, adding group by to each query in a union produces completely weird result (count(1) is 309 for each key, whereas it should be 1 and the "logical" incorrect value if internal count is lost is 500)- Nm, that groups by table column called key, which is weird but is what Hive does


    --
    This message was sent by Atlassian JIRA
    (v6.3.4#6332)
  • Hive QA (JIRA) at Oct 1, 2014 at 10:32 pm
    [ https://issues.apache.org/jira/browse/HIVE-8225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14155676#comment-14155676 ]

    Hive QA commented on HIVE-8225:
    -------------------------------



    {color:red}Overall{color}: -1 at least one tests failed

    Here are the results of testing the latest attachment:
    https://issues.apache.org/jira/secure/attachment/12672404/HIVE-8225.3.patch

    {color:red}ERROR:{color} -1 due to 14 failed/errored test(s), 6501 tests executed
    *Failed tests:*
    {noformat}
    org.apache.hive.hcatalog.pig.TestHCatLoader.testColumnarStorePushdown[5]
    org.apache.hive.hcatalog.pig.TestHCatLoader.testConvertBooleanToInt[5]
    org.apache.hive.hcatalog.pig.TestHCatLoader.testGetInputBytes[5]
    org.apache.hive.hcatalog.pig.TestHCatLoader.testProjectionsBasic[5]
    org.apache.hive.hcatalog.pig.TestHCatLoader.testReadDataBasic[5]
    org.apache.hive.hcatalog.pig.TestHCatLoader.testReadDataPrimitiveTypes[5]
    org.apache.hive.hcatalog.pig.TestHCatLoader.testReadPartitionedBasic[5]
    org.apache.hive.hcatalog.pig.TestHCatLoader.testSchemaLoadBasic[5]
    org.apache.hive.hcatalog.pig.TestHCatLoader.testSchemaLoadComplex[5]
    org.apache.hive.hcatalog.pig.TestHCatLoader.testSchemaLoadPrimitiveTypes[5]
    org.apache.hive.hcatalog.pig.TestHCatStorer.testDynamicPartitioningMultiPartColsNoDataInDataNoSpec[5]
    org.apache.hive.hcatalog.pig.TestHCatStorer.testEmptyStore[5]
    org.apache.hive.hcatalog.pig.TestHCatStorer.testNoAlias[5]
    org.apache.hive.hcatalog.pig.TestHCatStorer.testPartitionPublish[5]
    {noformat}

    Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/1076/testReport
    Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/1076/console
    Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-1076/

    Messages:
    {noformat}
    Executing org.apache.hive.ptest.execution.PrepPhase
    Executing org.apache.hive.ptest.execution.ExecutionPhase
    Executing org.apache.hive.ptest.execution.ReportingPhase
    Tests exited with: TestsFailedException: 14 tests failed
    {noformat}

    This message is automatically generated.

    ATTACHMENT ID: 12672404
    CBO trunk merge: union11 test fails due to incorrect plan
    ---------------------------------------------------------

    Key: HIVE-8225
    URL: https://issues.apache.org/jira/browse/HIVE-8225
    Project: Hive
    Issue Type: Bug
    Reporter: Sergey Shelukhin
    Assignee: Sergey Shelukhin
    Priority: Critical
    Fix For: 0.14.0

    Attachments: HIVE-8225.1.patch, HIVE-8225.2.patch, HIVE-8225.3.patch, HIVE-8225.inprogress.patch, HIVE-8225.inprogress.patch, HIVE-8225.patch


    The result changes to as if the union didn't have count() inside. The issue can be fixed by using srcunion.value outside the subquery in count (replace count(1) with count(srcunion.value)). Otherwise, it looks like count(1) node from union-ed queries is not present in AST at all, which might cause this result.
    -Interestingly, adding group by to each query in a union produces completely weird result (count(1) is 309 for each key, whereas it should be 1 and the "logical" incorrect value if internal count is lost is 500)- Nm, that groups by table column called key, which is weird but is what Hive does


    --
    This message was sent by Atlassian JIRA
    (v6.3.4#6332)
  • Pengcheng Xiong (JIRA) at Oct 1, 2014 at 10:42 pm
    [ https://issues.apache.org/jira/browse/HIVE-8225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14155701#comment-14155701 ]

    Pengcheng Xiong commented on HIVE-8225:
    ---------------------------------------

    The failed tests are not related to this patch.
    CBO trunk merge: union11 test fails due to incorrect plan
    ---------------------------------------------------------

    Key: HIVE-8225
    URL: https://issues.apache.org/jira/browse/HIVE-8225
    Project: Hive
    Issue Type: Bug
    Reporter: Sergey Shelukhin
    Assignee: Sergey Shelukhin
    Priority: Critical
    Fix For: 0.14.0

    Attachments: HIVE-8225.1.patch, HIVE-8225.2.patch, HIVE-8225.3.patch, HIVE-8225.inprogress.patch, HIVE-8225.inprogress.patch, HIVE-8225.patch


    The result changes to as if the union didn't have count() inside. The issue can be fixed by using srcunion.value outside the subquery in count (replace count(1) with count(srcunion.value)). Otherwise, it looks like count(1) node from union-ed queries is not present in AST at all, which might cause this result.
    -Interestingly, adding group by to each query in a union produces completely weird result (count(1) is 309 for each key, whereas it should be 1 and the "logical" incorrect value if internal count is lost is 500)- Nm, that groups by table column called key, which is weird but is what Hive does


    --
    This message was sent by Atlassian JIRA
    (v6.3.4#6332)
  • Pengcheng Xiong (JIRA) at Oct 2, 2014 at 8:18 pm
    [ https://issues.apache.org/jira/browse/HIVE-8225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14157108#comment-14157108 ]

    Pengcheng Xiong commented on HIVE-8225:
    ---------------------------------------

    I got BUILD SUCCESS when I run it on my local laptop. Anyway, I will resubmit my patch.
    CBO trunk merge: union11 test fails due to incorrect plan
    ---------------------------------------------------------

    Key: HIVE-8225
    URL: https://issues.apache.org/jira/browse/HIVE-8225
    Project: Hive
    Issue Type: Bug
    Reporter: Sergey Shelukhin
    Assignee: Sergey Shelukhin
    Priority: Critical
    Fix For: 0.14.0

    Attachments: HIVE-8225.1.patch, HIVE-8225.2.patch, HIVE-8225.3.patch, HIVE-8225.inprogress.patch, HIVE-8225.inprogress.patch, HIVE-8225.patch


    The result changes to as if the union didn't have count() inside. The issue can be fixed by using srcunion.value outside the subquery in count (replace count(1) with count(srcunion.value)). Otherwise, it looks like count(1) node from union-ed queries is not present in AST at all, which might cause this result.
    -Interestingly, adding group by to each query in a union produces completely weird result (count(1) is 309 for each key, whereas it should be 1 and the "logical" incorrect value if internal count is lost is 500)- Nm, that groups by table column called key, which is weird but is what Hive does


    --
    This message was sent by Atlassian JIRA
    (v6.3.4#6332)
  • Pengcheng Xiong (JIRA) at Oct 2, 2014 at 8:20 pm
    [ https://issues.apache.org/jira/browse/HIVE-8225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Pengcheng Xiong updated HIVE-8225:
    ----------------------------------
         Status: Open (was: Patch Available)
    CBO trunk merge: union11 test fails due to incorrect plan
    ---------------------------------------------------------

    Key: HIVE-8225
    URL: https://issues.apache.org/jira/browse/HIVE-8225
    Project: Hive
    Issue Type: Bug
    Reporter: Sergey Shelukhin
    Assignee: Sergey Shelukhin
    Priority: Critical
    Fix For: 0.14.0

    Attachments: HIVE-8225.1.patch, HIVE-8225.2.patch, HIVE-8225.3.patch, HIVE-8225.4.patch, HIVE-8225.inprogress.patch, HIVE-8225.inprogress.patch, HIVE-8225.patch


    The result changes to as if the union didn't have count() inside. The issue can be fixed by using srcunion.value outside the subquery in count (replace count(1) with count(srcunion.value)). Otherwise, it looks like count(1) node from union-ed queries is not present in AST at all, which might cause this result.
    -Interestingly, adding group by to each query in a union produces completely weird result (count(1) is 309 for each key, whereas it should be 1 and the "logical" incorrect value if internal count is lost is 500)- Nm, that groups by table column called key, which is weird but is what Hive does


    --
    This message was sent by Atlassian JIRA
    (v6.3.4#6332)
  • Pengcheng Xiong (JIRA) at Oct 2, 2014 at 8:20 pm
    [ https://issues.apache.org/jira/browse/HIVE-8225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Pengcheng Xiong updated HIVE-8225:
    ----------------------------------
         Attachment: HIVE-8225.4.patch
    CBO trunk merge: union11 test fails due to incorrect plan
    ---------------------------------------------------------

    Key: HIVE-8225
    URL: https://issues.apache.org/jira/browse/HIVE-8225
    Project: Hive
    Issue Type: Bug
    Reporter: Sergey Shelukhin
    Assignee: Sergey Shelukhin
    Priority: Critical
    Fix For: 0.14.0

    Attachments: HIVE-8225.1.patch, HIVE-8225.2.patch, HIVE-8225.3.patch, HIVE-8225.4.patch, HIVE-8225.inprogress.patch, HIVE-8225.inprogress.patch, HIVE-8225.patch


    The result changes to as if the union didn't have count() inside. The issue can be fixed by using srcunion.value outside the subquery in count (replace count(1) with count(srcunion.value)). Otherwise, it looks like count(1) node from union-ed queries is not present in AST at all, which might cause this result.
    -Interestingly, adding group by to each query in a union produces completely weird result (count(1) is 309 for each key, whereas it should be 1 and the "logical" incorrect value if internal count is lost is 500)- Nm, that groups by table column called key, which is weird but is what Hive does


    --
    This message was sent by Atlassian JIRA
    (v6.3.4#6332)
  • Pengcheng Xiong (JIRA) at Oct 2, 2014 at 8:20 pm
    [ https://issues.apache.org/jira/browse/HIVE-8225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Pengcheng Xiong updated HIVE-8225:
    ----------------------------------
         Status: Patch Available (was: Open)
    CBO trunk merge: union11 test fails due to incorrect plan
    ---------------------------------------------------------

    Key: HIVE-8225
    URL: https://issues.apache.org/jira/browse/HIVE-8225
    Project: Hive
    Issue Type: Bug
    Reporter: Sergey Shelukhin
    Assignee: Sergey Shelukhin
    Priority: Critical
    Fix For: 0.14.0

    Attachments: HIVE-8225.1.patch, HIVE-8225.2.patch, HIVE-8225.3.patch, HIVE-8225.4.patch, HIVE-8225.inprogress.patch, HIVE-8225.inprogress.patch, HIVE-8225.patch


    The result changes to as if the union didn't have count() inside. The issue can be fixed by using srcunion.value outside the subquery in count (replace count(1) with count(srcunion.value)). Otherwise, it looks like count(1) node from union-ed queries is not present in AST at all, which might cause this result.
    -Interestingly, adding group by to each query in a union produces completely weird result (count(1) is 309 for each key, whereas it should be 1 and the "logical" incorrect value if internal count is lost is 500)- Nm, that groups by table column called key, which is weird but is what Hive does


    --
    This message was sent by Atlassian JIRA
    (v6.3.4#6332)
  • Hive QA (JIRA) at Oct 3, 2014 at 8:23 am
    [ https://issues.apache.org/jira/browse/HIVE-8225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14157791#comment-14157791 ]

    Hive QA commented on HIVE-8225:
    -------------------------------



    {color:red}Overall{color}: -1 at least one tests failed

    Here are the results of testing the latest attachment:
    https://issues.apache.org/jira/secure/attachment/12672621/HIVE-8225.4.patch

    {color:red}ERROR:{color} -1 due to 14 failed/errored test(s), 6541 tests executed
    *Failed tests:*
    {noformat}
    org.apache.hive.hcatalog.pig.TestHCatLoader.testColumnarStorePushdown[5]
    org.apache.hive.hcatalog.pig.TestHCatLoader.testConvertBooleanToInt[5]
    org.apache.hive.hcatalog.pig.TestHCatLoader.testGetInputBytes[5]
    org.apache.hive.hcatalog.pig.TestHCatLoader.testProjectionsBasic[5]
    org.apache.hive.hcatalog.pig.TestHCatLoader.testReadDataBasic[5]
    org.apache.hive.hcatalog.pig.TestHCatLoader.testReadDataPrimitiveTypes[5]
    org.apache.hive.hcatalog.pig.TestHCatLoader.testReadPartitionedBasic[5]
    org.apache.hive.hcatalog.pig.TestHCatLoader.testSchemaLoadBasic[5]
    org.apache.hive.hcatalog.pig.TestHCatLoader.testSchemaLoadComplex[5]
    org.apache.hive.hcatalog.pig.TestHCatLoader.testSchemaLoadPrimitiveTypes[5]
    org.apache.hive.hcatalog.pig.TestHCatStorer.testDynamicPartitioningMultiPartColsNoDataInDataNoSpec[5]
    org.apache.hive.hcatalog.pig.TestHCatStorer.testEmptyStore[5]
    org.apache.hive.hcatalog.pig.TestHCatStorer.testNoAlias[5]
    org.apache.hive.hcatalog.pig.TestHCatStorer.testPartitionPublish[5]
    {noformat}

    Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/1096/testReport
    Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/1096/console
    Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-1096/

    Messages:
    {noformat}
    Executing org.apache.hive.ptest.execution.PrepPhase
    Executing org.apache.hive.ptest.execution.ExecutionPhase
    Executing org.apache.hive.ptest.execution.ReportingPhase
    Tests exited with: TestsFailedException: 14 tests failed
    {noformat}

    This message is automatically generated.

    ATTACHMENT ID: 12672621
    CBO trunk merge: union11 test fails due to incorrect plan
    ---------------------------------------------------------

    Key: HIVE-8225
    URL: https://issues.apache.org/jira/browse/HIVE-8225
    Project: Hive
    Issue Type: Bug
    Reporter: Sergey Shelukhin
    Assignee: Sergey Shelukhin
    Priority: Critical
    Fix For: 0.14.0

    Attachments: HIVE-8225.1.patch, HIVE-8225.2.patch, HIVE-8225.3.patch, HIVE-8225.4.patch, HIVE-8225.inprogress.patch, HIVE-8225.inprogress.patch, HIVE-8225.patch


    The result changes to as if the union didn't have count() inside. The issue can be fixed by using srcunion.value outside the subquery in count (replace count(1) with count(srcunion.value)). Otherwise, it looks like count(1) node from union-ed queries is not present in AST at all, which might cause this result.
    -Interestingly, adding group by to each query in a union produces completely weird result (count(1) is 309 for each key, whereas it should be 1 and the "logical" incorrect value if internal count is lost is 500)- Nm, that groups by table column called key, which is weird but is what Hive does


    --
    This message was sent by Atlassian JIRA
    (v6.3.4#6332)
  • Pengcheng Xiong (JIRA) at Oct 6, 2014 at 5:39 pm
    [ https://issues.apache.org/jira/browse/HIVE-8225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Pengcheng Xiong updated HIVE-8225:
    ----------------------------------
         Status: Open (was: Patch Available)
    CBO trunk merge: union11 test fails due to incorrect plan
    ---------------------------------------------------------

    Key: HIVE-8225
    URL: https://issues.apache.org/jira/browse/HIVE-8225
    Project: Hive
    Issue Type: Bug
    Reporter: Sergey Shelukhin
    Assignee: Sergey Shelukhin
    Priority: Critical
    Fix For: 0.14.0

    Attachments: HIVE-8225.1.patch, HIVE-8225.2.patch, HIVE-8225.3.patch, HIVE-8225.4.patch, HIVE-8225.5.patch, HIVE-8225.inprogress.patch, HIVE-8225.inprogress.patch, HIVE-8225.patch


    The result changes to as if the union didn't have count() inside. The issue can be fixed by using srcunion.value outside the subquery in count (replace count(1) with count(srcunion.value)). Otherwise, it looks like count(1) node from union-ed queries is not present in AST at all, which might cause this result.
    -Interestingly, adding group by to each query in a union produces completely weird result (count(1) is 309 for each key, whereas it should be 1 and the "logical" incorrect value if internal count is lost is 500)- Nm, that groups by table column called key, which is weird but is what Hive does


    --
    This message was sent by Atlassian JIRA
    (v6.3.4#6332)
  • Pengcheng Xiong (JIRA) at Oct 6, 2014 at 5:39 pm
    [ https://issues.apache.org/jira/browse/HIVE-8225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Pengcheng Xiong updated HIVE-8225:
    ----------------------------------
         Attachment: HIVE-8225.5.patch
    CBO trunk merge: union11 test fails due to incorrect plan
    ---------------------------------------------------------

    Key: HIVE-8225
    URL: https://issues.apache.org/jira/browse/HIVE-8225
    Project: Hive
    Issue Type: Bug
    Reporter: Sergey Shelukhin
    Assignee: Sergey Shelukhin
    Priority: Critical
    Fix For: 0.14.0

    Attachments: HIVE-8225.1.patch, HIVE-8225.2.patch, HIVE-8225.3.patch, HIVE-8225.4.patch, HIVE-8225.5.patch, HIVE-8225.inprogress.patch, HIVE-8225.inprogress.patch, HIVE-8225.patch


    The result changes to as if the union didn't have count() inside. The issue can be fixed by using srcunion.value outside the subquery in count (replace count(1) with count(srcunion.value)). Otherwise, it looks like count(1) node from union-ed queries is not present in AST at all, which might cause this result.
    -Interestingly, adding group by to each query in a union produces completely weird result (count(1) is 309 for each key, whereas it should be 1 and the "logical" incorrect value if internal count is lost is 500)- Nm, that groups by table column called key, which is weird but is what Hive does


    --
    This message was sent by Atlassian JIRA
    (v6.3.4#6332)
  • Pengcheng Xiong (JIRA) at Oct 6, 2014 at 5:39 pm
    [ https://issues.apache.org/jira/browse/HIVE-8225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Pengcheng Xiong updated HIVE-8225:
    ----------------------------------
         Status: Patch Available (was: Open)
    CBO trunk merge: union11 test fails due to incorrect plan
    ---------------------------------------------------------

    Key: HIVE-8225
    URL: https://issues.apache.org/jira/browse/HIVE-8225
    Project: Hive
    Issue Type: Bug
    Reporter: Sergey Shelukhin
    Assignee: Sergey Shelukhin
    Priority: Critical
    Fix For: 0.14.0

    Attachments: HIVE-8225.1.patch, HIVE-8225.2.patch, HIVE-8225.3.patch, HIVE-8225.4.patch, HIVE-8225.5.patch, HIVE-8225.inprogress.patch, HIVE-8225.inprogress.patch, HIVE-8225.patch


    The result changes to as if the union didn't have count() inside. The issue can be fixed by using srcunion.value outside the subquery in count (replace count(1) with count(srcunion.value)). Otherwise, it looks like count(1) node from union-ed queries is not present in AST at all, which might cause this result.
    -Interestingly, adding group by to each query in a union produces completely weird result (count(1) is 309 for each key, whereas it should be 1 and the "logical" incorrect value if internal count is lost is 500)- Nm, that groups by table column called key, which is weird but is what Hive does


    --
    This message was sent by Atlassian JIRA
    (v6.3.4#6332)
  • Hive QA (JIRA) at Oct 6, 2014 at 11:19 pm
    [ https://issues.apache.org/jira/browse/HIVE-8225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14161191#comment-14161191 ]

    Hive QA commented on HIVE-8225:
    -------------------------------



    {color:green}Overall{color}: +1 all checks pass

    Here are the results of testing the latest attachment:
    https://issues.apache.org/jira/secure/attachment/12673134/HIVE-8225.5.patch

    {color:green}SUCCESS:{color} +1 6525 tests passed

    Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/1134/testReport
    Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/1134/console
    Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-1134/

    Messages:
    {noformat}
    Executing org.apache.hive.ptest.execution.PrepPhase
    Executing org.apache.hive.ptest.execution.ExecutionPhase
    Executing org.apache.hive.ptest.execution.ReportingPhase
    {noformat}

    This message is automatically generated.

    ATTACHMENT ID: 12673134
    CBO trunk merge: union11 test fails due to incorrect plan
    ---------------------------------------------------------

    Key: HIVE-8225
    URL: https://issues.apache.org/jira/browse/HIVE-8225
    Project: Hive
    Issue Type: Bug
    Reporter: Sergey Shelukhin
    Assignee: Sergey Shelukhin
    Priority: Critical
    Fix For: 0.14.0

    Attachments: HIVE-8225.1.patch, HIVE-8225.2.patch, HIVE-8225.3.patch, HIVE-8225.4.patch, HIVE-8225.5.patch, HIVE-8225.inprogress.patch, HIVE-8225.inprogress.patch, HIVE-8225.patch


    The result changes to as if the union didn't have count() inside. The issue can be fixed by using srcunion.value outside the subquery in count (replace count(1) with count(srcunion.value)). Otherwise, it looks like count(1) node from union-ed queries is not present in AST at all, which might cause this result.
    -Interestingly, adding group by to each query in a union produces completely weird result (count(1) is 309 for each key, whereas it should be 1 and the "logical" incorrect value if internal count is lost is 500)- Nm, that groups by table column called key, which is weird but is what Hive does


    --
    This message was sent by Atlassian JIRA
    (v6.3.4#6332)
  • Pengcheng Xiong (JIRA) at Oct 6, 2014 at 11:34 pm
    [ https://issues.apache.org/jira/browse/HIVE-8225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14161213#comment-14161213 ]

    Pengcheng Xiong commented on HIVE-8225:
    ---------------------------------------

    [~ashutoshc] could you please kindly commit? This will also solve union5 and union7 problem.
    CBO trunk merge: union11 test fails due to incorrect plan
    ---------------------------------------------------------

    Key: HIVE-8225
    URL: https://issues.apache.org/jira/browse/HIVE-8225
    Project: Hive
    Issue Type: Bug
    Reporter: Sergey Shelukhin
    Assignee: Sergey Shelukhin
    Priority: Critical
    Fix For: 0.14.0

    Attachments: HIVE-8225.1.patch, HIVE-8225.2.patch, HIVE-8225.3.patch, HIVE-8225.4.patch, HIVE-8225.5.patch, HIVE-8225.inprogress.patch, HIVE-8225.inprogress.patch, HIVE-8225.patch


    The result changes to as if the union didn't have count() inside. The issue can be fixed by using srcunion.value outside the subquery in count (replace count(1) with count(srcunion.value)). Otherwise, it looks like count(1) node from union-ed queries is not present in AST at all, which might cause this result.
    -Interestingly, adding group by to each query in a union produces completely weird result (count(1) is 309 for each key, whereas it should be 1 and the "logical" incorrect value if internal count is lost is 500)- Nm, that groups by table column called key, which is weird but is what Hive does


    --
    This message was sent by Atlassian JIRA
    (v6.3.4#6332)
  • Sergey Shelukhin (JIRA) at Oct 7, 2014 at 9:43 pm
    [ https://issues.apache.org/jira/browse/HIVE-8225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14162597#comment-14162597 ]

    Sergey Shelukhin commented on HIVE-8225:
    ----------------------------------------

    Btw, can you try the original simple query
    {noformat}
    select unionsrc.key FROM (select 'tst1' as key, count(1) as value from src s1) unionsrc;
    {noformat}
    Just checking. I thought derived projection we need has to be different (one should include the dummy count() for it to be accounted for and work, and one should exclude it because the query doesn't have it); I wonder if this will work with simple query?
    CBO trunk merge: union11 test fails due to incorrect plan
    ---------------------------------------------------------

    Key: HIVE-8225
    URL: https://issues.apache.org/jira/browse/HIVE-8225
    Project: Hive
    Issue Type: Bug
    Reporter: Sergey Shelukhin
    Assignee: Sergey Shelukhin
    Priority: Critical
    Fix For: 0.14.0

    Attachments: HIVE-8225.1.patch, HIVE-8225.2.patch, HIVE-8225.3.patch, HIVE-8225.4.patch, HIVE-8225.5.patch, HIVE-8225.inprogress.patch, HIVE-8225.inprogress.patch, HIVE-8225.patch


    The result changes to as if the union didn't have count() inside. The issue can be fixed by using srcunion.value outside the subquery in count (replace count(1) with count(srcunion.value)). Otherwise, it looks like count(1) node from union-ed queries is not present in AST at all, which might cause this result.
    -Interestingly, adding group by to each query in a union produces completely weird result (count(1) is 309 for each key, whereas it should be 1 and the "logical" incorrect value if internal count is lost is 500)- Nm, that groups by table column called key, which is weird but is what Hive does


    --
    This message was sent by Atlassian JIRA
    (v6.3.4#6332)
  • Sergey Shelukhin (JIRA) at Oct 7, 2014 at 9:44 pm
    [ https://issues.apache.org/jira/browse/HIVE-8225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14162598#comment-14162598 ]

    Sergey Shelukhin commented on HIVE-8225:
    ----------------------------------------

    Also some feedback in RB, mostly nits
    CBO trunk merge: union11 test fails due to incorrect plan
    ---------------------------------------------------------

    Key: HIVE-8225
    URL: https://issues.apache.org/jira/browse/HIVE-8225
    Project: Hive
    Issue Type: Bug
    Reporter: Sergey Shelukhin
    Assignee: Sergey Shelukhin
    Priority: Critical
    Fix For: 0.14.0

    Attachments: HIVE-8225.1.patch, HIVE-8225.2.patch, HIVE-8225.3.patch, HIVE-8225.4.patch, HIVE-8225.5.patch, HIVE-8225.inprogress.patch, HIVE-8225.inprogress.patch, HIVE-8225.patch


    The result changes to as if the union didn't have count() inside. The issue can be fixed by using srcunion.value outside the subquery in count (replace count(1) with count(srcunion.value)). Otherwise, it looks like count(1) node from union-ed queries is not present in AST at all, which might cause this result.
    -Interestingly, adding group by to each query in a union produces completely weird result (count(1) is 309 for each key, whereas it should be 1 and the "logical" incorrect value if internal count is lost is 500)- Nm, that groups by table column called key, which is weird but is what Hive does


    --
    This message was sent by Atlassian JIRA
    (v6.3.4#6332)
  • Pengcheng Xiong (JIRA) at Oct 7, 2014 at 11:51 pm
    [ https://issues.apache.org/jira/browse/HIVE-8225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14162802#comment-14162802 ]

    Pengcheng Xiong commented on HIVE-8225:
    ---------------------------------------

    [~sershe], I have just tried select unionsrc.key FROM (select 'tst1' as key, count(1) as value from src s1) unionsrc;
    the output is "tst1" as we expect.
    CBO trunk merge: union11 test fails due to incorrect plan
    ---------------------------------------------------------

    Key: HIVE-8225
    URL: https://issues.apache.org/jira/browse/HIVE-8225
    Project: Hive
    Issue Type: Bug
    Reporter: Sergey Shelukhin
    Assignee: Sergey Shelukhin
    Priority: Critical
    Fix For: 0.14.0

    Attachments: HIVE-8225.1.patch, HIVE-8225.2.patch, HIVE-8225.3.patch, HIVE-8225.4.patch, HIVE-8225.5.patch, HIVE-8225.inprogress.patch, HIVE-8225.inprogress.patch, HIVE-8225.patch


    The result changes to as if the union didn't have count() inside. The issue can be fixed by using srcunion.value outside the subquery in count (replace count(1) with count(srcunion.value)). Otherwise, it looks like count(1) node from union-ed queries is not present in AST at all, which might cause this result.
    -Interestingly, adding group by to each query in a union produces completely weird result (count(1) is 309 for each key, whereas it should be 1 and the "logical" incorrect value if internal count is lost is 500)- Nm, that groups by table column called key, which is weird but is what Hive does


    --
    This message was sent by Atlassian JIRA
    (v6.3.4#6332)
  • Pengcheng Xiong (JIRA) at Oct 8, 2014 at 12:10 am
    [ https://issues.apache.org/jira/browse/HIVE-8225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Pengcheng Xiong updated HIVE-8225:
    ----------------------------------
         Status: Open (was: Patch Available)
    CBO trunk merge: union11 test fails due to incorrect plan
    ---------------------------------------------------------

    Key: HIVE-8225
    URL: https://issues.apache.org/jira/browse/HIVE-8225
    Project: Hive
    Issue Type: Bug
    Reporter: Sergey Shelukhin
    Assignee: Sergey Shelukhin
    Priority: Critical
    Fix For: 0.14.0

    Attachments: HIVE-8225.1.patch, HIVE-8225.2.patch, HIVE-8225.3.patch, HIVE-8225.4.patch, HIVE-8225.5.patch, HIVE-8225.6.patch, HIVE-8225.inprogress.patch, HIVE-8225.inprogress.patch, HIVE-8225.patch


    The result changes to as if the union didn't have count() inside. The issue can be fixed by using srcunion.value outside the subquery in count (replace count(1) with count(srcunion.value)). Otherwise, it looks like count(1) node from union-ed queries is not present in AST at all, which might cause this result.
    -Interestingly, adding group by to each query in a union produces completely weird result (count(1) is 309 for each key, whereas it should be 1 and the "logical" incorrect value if internal count is lost is 500)- Nm, that groups by table column called key, which is weird but is what Hive does


    --
    This message was sent by Atlassian JIRA
    (v6.3.4#6332)
  • Pengcheng Xiong (JIRA) at Oct 8, 2014 at 12:10 am
    [ https://issues.apache.org/jira/browse/HIVE-8225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Pengcheng Xiong updated HIVE-8225:
    ----------------------------------
         Attachment: HIVE-8225.6.patch

    address Sergey's comments
    CBO trunk merge: union11 test fails due to incorrect plan
    ---------------------------------------------------------

    Key: HIVE-8225
    URL: https://issues.apache.org/jira/browse/HIVE-8225
    Project: Hive
    Issue Type: Bug
    Reporter: Sergey Shelukhin
    Assignee: Sergey Shelukhin
    Priority: Critical
    Fix For: 0.14.0

    Attachments: HIVE-8225.1.patch, HIVE-8225.2.patch, HIVE-8225.3.patch, HIVE-8225.4.patch, HIVE-8225.5.patch, HIVE-8225.6.patch, HIVE-8225.inprogress.patch, HIVE-8225.inprogress.patch, HIVE-8225.patch


    The result changes to as if the union didn't have count() inside. The issue can be fixed by using srcunion.value outside the subquery in count (replace count(1) with count(srcunion.value)). Otherwise, it looks like count(1) node from union-ed queries is not present in AST at all, which might cause this result.
    -Interestingly, adding group by to each query in a union produces completely weird result (count(1) is 309 for each key, whereas it should be 1 and the "logical" incorrect value if internal count is lost is 500)- Nm, that groups by table column called key, which is weird but is what Hive does


    --
    This message was sent by Atlassian JIRA
    (v6.3.4#6332)
  • Pengcheng Xiong (JIRA) at Oct 8, 2014 at 12:11 am
    [ https://issues.apache.org/jira/browse/HIVE-8225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Pengcheng Xiong updated HIVE-8225:
    ----------------------------------
         Status: Patch Available (was: Open)
    CBO trunk merge: union11 test fails due to incorrect plan
    ---------------------------------------------------------

    Key: HIVE-8225
    URL: https://issues.apache.org/jira/browse/HIVE-8225
    Project: Hive
    Issue Type: Bug
    Reporter: Sergey Shelukhin
    Assignee: Sergey Shelukhin
    Priority: Critical
    Fix For: 0.14.0

    Attachments: HIVE-8225.1.patch, HIVE-8225.2.patch, HIVE-8225.3.patch, HIVE-8225.4.patch, HIVE-8225.5.patch, HIVE-8225.6.patch, HIVE-8225.inprogress.patch, HIVE-8225.inprogress.patch, HIVE-8225.patch


    The result changes to as if the union didn't have count() inside. The issue can be fixed by using srcunion.value outside the subquery in count (replace count(1) with count(srcunion.value)). Otherwise, it looks like count(1) node from union-ed queries is not present in AST at all, which might cause this result.
    -Interestingly, adding group by to each query in a union produces completely weird result (count(1) is 309 for each key, whereas it should be 1 and the "logical" incorrect value if internal count is lost is 500)- Nm, that groups by table column called key, which is weird but is what Hive does


    --
    This message was sent by Atlassian JIRA
    (v6.3.4#6332)
  • Pengcheng Xiong (JIRA) at Oct 8, 2014 at 5:15 am
    [ https://issues.apache.org/jira/browse/HIVE-8225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Pengcheng Xiong updated HIVE-8225:
    ----------------------------------
         Status: Open (was: Patch Available)
    CBO trunk merge: union11 test fails due to incorrect plan
    ---------------------------------------------------------

    Key: HIVE-8225
    URL: https://issues.apache.org/jira/browse/HIVE-8225
    Project: Hive
    Issue Type: Bug
    Reporter: Sergey Shelukhin
    Assignee: Sergey Shelukhin
    Priority: Critical
    Fix For: 0.14.0

    Attachments: HIVE-8225.1.patch, HIVE-8225.2.patch, HIVE-8225.3.patch, HIVE-8225.4.patch, HIVE-8225.5.patch, HIVE-8225.6.patch, HIVE-8225.inprogress.patch, HIVE-8225.inprogress.patch, HIVE-8225.patch


    The result changes to as if the union didn't have count() inside. The issue can be fixed by using srcunion.value outside the subquery in count (replace count(1) with count(srcunion.value)). Otherwise, it looks like count(1) node from union-ed queries is not present in AST at all, which might cause this result.
    -Interestingly, adding group by to each query in a union produces completely weird result (count(1) is 309 for each key, whereas it should be 1 and the "logical" incorrect value if internal count is lost is 500)- Nm, that groups by table column called key, which is weird but is what Hive does


    --
    This message was sent by Atlassian JIRA
    (v6.3.4#6332)
  • Pengcheng Xiong (JIRA) at Oct 8, 2014 at 5:16 am
    [ https://issues.apache.org/jira/browse/HIVE-8225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Pengcheng Xiong updated HIVE-8225:
    ----------------------------------
         Attachment: HIVE-8225.7.patch

    address John's comments
    CBO trunk merge: union11 test fails due to incorrect plan
    ---------------------------------------------------------

    Key: HIVE-8225
    URL: https://issues.apache.org/jira/browse/HIVE-8225
    Project: Hive
    Issue Type: Bug
    Reporter: Sergey Shelukhin
    Assignee: Sergey Shelukhin
    Priority: Critical
    Fix For: 0.14.0

    Attachments: HIVE-8225.1.patch, HIVE-8225.2.patch, HIVE-8225.3.patch, HIVE-8225.4.patch, HIVE-8225.5.patch, HIVE-8225.6.patch, HIVE-8225.7.patch, HIVE-8225.inprogress.patch, HIVE-8225.inprogress.patch, HIVE-8225.patch


    The result changes to as if the union didn't have count() inside. The issue can be fixed by using srcunion.value outside the subquery in count (replace count(1) with count(srcunion.value)). Otherwise, it looks like count(1) node from union-ed queries is not present in AST at all, which might cause this result.
    -Interestingly, adding group by to each query in a union produces completely weird result (count(1) is 309 for each key, whereas it should be 1 and the "logical" incorrect value if internal count is lost is 500)- Nm, that groups by table column called key, which is weird but is what Hive does


    --
    This message was sent by Atlassian JIRA
    (v6.3.4#6332)
  • Pengcheng Xiong (JIRA) at Oct 8, 2014 at 5:16 am
    [ https://issues.apache.org/jira/browse/HIVE-8225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Pengcheng Xiong updated HIVE-8225:
    ----------------------------------
         Status: Patch Available (was: Open)
    CBO trunk merge: union11 test fails due to incorrect plan
    ---------------------------------------------------------

    Key: HIVE-8225
    URL: https://issues.apache.org/jira/browse/HIVE-8225
    Project: Hive
    Issue Type: Bug
    Reporter: Sergey Shelukhin
    Assignee: Sergey Shelukhin
    Priority: Critical
    Fix For: 0.14.0

    Attachments: HIVE-8225.1.patch, HIVE-8225.2.patch, HIVE-8225.3.patch, HIVE-8225.4.patch, HIVE-8225.5.patch, HIVE-8225.6.patch, HIVE-8225.7.patch, HIVE-8225.inprogress.patch, HIVE-8225.inprogress.patch, HIVE-8225.patch


    The result changes to as if the union didn't have count() inside. The issue can be fixed by using srcunion.value outside the subquery in count (replace count(1) with count(srcunion.value)). Otherwise, it looks like count(1) node from union-ed queries is not present in AST at all, which might cause this result.
    -Interestingly, adding group by to each query in a union produces completely weird result (count(1) is 309 for each key, whereas it should be 1 and the "logical" incorrect value if internal count is lost is 500)- Nm, that groups by table column called key, which is weird but is what Hive does


    --
    This message was sent by Atlassian JIRA
    (v6.3.4#6332)
  • Hive QA (JIRA) at Oct 8, 2014 at 5:40 pm
    [ https://issues.apache.org/jira/browse/HIVE-8225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14163837#comment-14163837 ]

    Hive QA commented on HIVE-8225:
    -------------------------------



    {color:red}Overall{color}: -1 no tests executed

    Here are the results of testing the latest attachment:
    https://issues.apache.org/jira/secure/attachment/12673527/HIVE-8225.7.patch

    Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/1169/testReport
    Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/1169/console
    Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-1169/

    Messages:
    {noformat}
    **** This message was trimmed, see log for full details ****
    warning(200): IdentifiersParser.g:68:4:
    Decision can match input such as "LPAREN LPAREN KW_CASE" using multiple alternatives: 1, 2

    As a result, alternative(s) 2 were disabled for that input
    warning(200): IdentifiersParser.g:68:4:
    Decision can match input such as "LPAREN KW_NOT KW_FALSE" using multiple alternatives: 1, 2

    As a result, alternative(s) 2 were disabled for that input
    warning(200): IdentifiersParser.g:68:4:
    Decision can match input such as "LPAREN KW_DATE StringLiteral" using multiple alternatives: 1, 2

    As a result, alternative(s) 2 were disabled for that input
    warning(200): IdentifiersParser.g:68:4:
    Decision can match input such as "LPAREN KW_NOT KW_TRUE" using multiple alternatives: 1, 2

    As a result, alternative(s) 2 were disabled for that input
    warning(200): IdentifiersParser.g:68:4:
    Decision can match input such as "LPAREN KW_NOT KW_MAP" using multiple alternatives: 1, 2

    As a result, alternative(s) 2 were disabled for that input
    warning(200): IdentifiersParser.g:68:4:
    Decision can match input such as "LPAREN KW_CASE KW_MAP" using multiple alternatives: 1, 2

    As a result, alternative(s) 2 were disabled for that input
    warning(200): IdentifiersParser.g:68:4:
    Decision can match input such as "LPAREN LPAREN KW_MAP" using multiple alternatives: 1, 2

    As a result, alternative(s) 2 were disabled for that input
    warning(200): IdentifiersParser.g:68:4:
    Decision can match input such as "LPAREN KW_CASE KW_UNIONTYPE" using multiple alternatives: 1, 2

    As a result, alternative(s) 2 were disabled for that input
    warning(200): IdentifiersParser.g:68:4:
    Decision can match input such as "LPAREN KW_CASE KW_STRUCT" using multiple alternatives: 1, 2

    As a result, alternative(s) 2 were disabled for that input
    warning(200): IdentifiersParser.g:68:4:
    Decision can match input such as "LPAREN KW_NOT KW_IF" using multiple alternatives: 1, 2

    As a result, alternative(s) 2 were disabled for that input
    warning(200): IdentifiersParser.g:68:4:
    Decision can match input such as "LPAREN KW_CASE KW_IF" using multiple alternatives: 1, 2

    As a result, alternative(s) 2 were disabled for that input
    warning(200): IdentifiersParser.g:68:4:
    Decision can match input such as "LPAREN LPAREN KW_IF" using multiple alternatives: 1, 2

    As a result, alternative(s) 2 were disabled for that input
    warning(200): IdentifiersParser.g:115:5:
    Decision can match input such as "KW_CLUSTER KW_BY LPAREN" using multiple alternatives: 1, 2

    As a result, alternative(s) 2 were disabled for that input
    warning(200): IdentifiersParser.g:127:5:
    Decision can match input such as "KW_PARTITION KW_BY LPAREN" using multiple alternatives: 1, 2

    As a result, alternative(s) 2 were disabled for that input
    warning(200): IdentifiersParser.g:138:5:
    Decision can match input such as "KW_DISTRIBUTE KW_BY LPAREN" using multiple alternatives: 1, 2

    As a result, alternative(s) 2 were disabled for that input
    warning(200): IdentifiersParser.g:149:5:
    Decision can match input such as "KW_SORT KW_BY LPAREN" using multiple alternatives: 1, 2

    As a result, alternative(s) 2 were disabled for that input
    warning(200): IdentifiersParser.g:166:7:
    Decision can match input such as "STAR" using multiple alternatives: 1, 2

    As a result, alternative(s) 2 were disabled for that input
    warning(200): IdentifiersParser.g:179:5:
    Decision can match input such as "KW_STRUCT" using multiple alternatives: 4, 6

    As a result, alternative(s) 6 were disabled for that input
    warning(200): IdentifiersParser.g:179:5:
    Decision can match input such as "KW_ARRAY" using multiple alternatives: 2, 6

    As a result, alternative(s) 6 were disabled for that input
    warning(200): IdentifiersParser.g:179:5:
    Decision can match input such as "KW_UNIONTYPE" using multiple alternatives: 5, 6

    As a result, alternative(s) 6 were disabled for that input
    warning(200): IdentifiersParser.g:261:5:
    Decision can match input such as "KW_TRUE" using multiple alternatives: 3, 8

    As a result, alternative(s) 8 were disabled for that input
    warning(200): IdentifiersParser.g:261:5:
    Decision can match input such as "KW_NULL" using multiple alternatives: 1, 8

    As a result, alternative(s) 8 were disabled for that input
    warning(200): IdentifiersParser.g:261:5:
    Decision can match input such as "KW_FALSE" using multiple alternatives: 3, 8

    As a result, alternative(s) 8 were disabled for that input
    warning(200): IdentifiersParser.g:261:5:
    Decision can match input such as "KW_DATE StringLiteral" using multiple alternatives: 2, 3

    As a result, alternative(s) 3 were disabled for that input
    warning(200): IdentifiersParser.g:393:5:
    Decision can match input such as "KW_BETWEEN KW_MAP LPAREN" using multiple alternatives: 8, 9

    As a result, alternative(s) 9 were disabled for that input
    warning(200): IdentifiersParser.g:393:5:
    Decision can match input such as "{KW_LIKE, KW_REGEXP, KW_RLIKE} KW_INSERT KW_INTO" using multiple alternatives: 2, 9

    As a result, alternative(s) 9 were disabled for that input
    warning(200): IdentifiersParser.g:393:5:
    Decision can match input such as "{KW_LIKE, KW_REGEXP, KW_RLIKE} KW_UNION KW_ALL" using multiple alternatives: 2, 9

    As a result, alternative(s) 9 were disabled for that input
    warning(200): IdentifiersParser.g:393:5:
    Decision can match input such as "{KW_LIKE, KW_REGEXP, KW_RLIKE} KW_SORT KW_BY" using multiple alternatives: 2, 9

    As a result, alternative(s) 9 were disabled for that input
    warning(200): IdentifiersParser.g:393:5:
    Decision can match input such as "{KW_LIKE, KW_REGEXP, KW_RLIKE} KW_CLUSTER KW_BY" using multiple alternatives: 2, 9

    As a result, alternative(s) 9 were disabled for that input
    warning(200): IdentifiersParser.g:393:5:
    Decision can match input such as "{KW_LIKE, KW_REGEXP, KW_RLIKE} KW_LATERAL KW_VIEW" using multiple alternatives: 2, 9

    As a result, alternative(s) 9 were disabled for that input
    warning(200): IdentifiersParser.g:393:5:
    Decision can match input such as "{KW_LIKE, KW_REGEXP, KW_RLIKE} KW_MAP LPAREN" using multiple alternatives: 2, 9

    As a result, alternative(s) 9 were disabled for that input
    warning(200): IdentifiersParser.g:393:5:
    Decision can match input such as "{KW_LIKE, KW_REGEXP, KW_RLIKE} KW_DISTRIBUTE KW_BY" using multiple alternatives: 2, 9

    As a result, alternative(s) 9 were disabled for that input
    warning(200): IdentifiersParser.g:393:5:
    Decision can match input such as "{KW_LIKE, KW_REGEXP, KW_RLIKE} KW_ORDER KW_BY" using multiple alternatives: 2, 9

    As a result, alternative(s) 9 were disabled for that input
    warning(200): IdentifiersParser.g:393:5:
    Decision can match input such as "{KW_LIKE, KW_REGEXP, KW_RLIKE} KW_GROUP KW_BY" using multiple alternatives: 2, 9

    As a result, alternative(s) 9 were disabled for that input
    warning(200): IdentifiersParser.g:393:5:
    Decision can match input such as "{KW_LIKE, KW_REGEXP, KW_RLIKE} KW_INSERT KW_OVERWRITE" using multiple alternatives: 2, 9

    As a result, alternative(s) 9 were disabled for that input
    warning(200): IdentifiersParser.g:518:5:
    Decision can match input such as "{AMPERSAND..BITWISEXOR, DIV..DIVIDE, EQUAL..EQUAL_NS, GREATERTHAN..GREATERTHANOREQUALTO, KW_AND, KW_ARRAY, KW_BETWEEN..KW_BOOLEAN, KW_CASE, KW_DOUBLE, KW_FLOAT, KW_IF, KW_IN, KW_INT, KW_LIKE, KW_MAP, KW_NOT, KW_OR, KW_REGEXP, KW_RLIKE, KW_SMALLINT, KW_STRING..KW_STRUCT, KW_TINYINT, KW_UNIONTYPE, KW_WHEN, LESSTHAN..LESSTHANOREQUALTO, MINUS..NOTEQUAL, PLUS, STAR, TILDE}" using multiple alternatives: 1, 3

    As a result, alternative(s) 3 were disabled for that input
    [INFO]
    [INFO] --- maven-remote-resources-plugin:1.5:process (default) @ hive-exec ---
    Downloading: http://www.datanucleus.org/downloads/maven2/net/hydromatic/linq4j/0.4/linq4j-0.4.pom
    [INFO] ------------------------------------------------------------------------
    [INFO] Reactor Summary:
    [INFO]
    [INFO] Hive .............................................. SUCCESS [13.751s]
    [INFO] Hive Shims Common ................................. SUCCESS [8.641s]
    [INFO] Hive Shims 0.20 ................................... SUCCESS [3.972s]
    [INFO] Hive Shims Secure Common .......................... SUCCESS [4.562s]
    [INFO] Hive Shims 0.20S .................................. SUCCESS [2.412s]
    [INFO] Hive Shims 0.23 ................................... SUCCESS [7.842s]
    [INFO] Hive Shims ........................................ SUCCESS [0.821s]
    [INFO] Hive Common ....................................... SUCCESS [9.242s]
    [INFO] Hive Serde ........................................ SUCCESS [16.169s]
    [INFO] Hive Metastore .................................... SUCCESS [37.591s]
    [INFO] Hive Ant Utilities ................................ SUCCESS [1.895s]
    [INFO] Hive Query Language ............................... FAILURE [33.290s]
    [INFO] Hive Service ...................................... SKIPPED
    [INFO] Hive Accumulo Handler ............................. SKIPPED
    [INFO] Hive JDBC ......................................... SKIPPED
    [INFO] Hive Beeline ...................................... SKIPPED
    [INFO] Hive CLI .......................................... SKIPPED
    [INFO] Hive Contrib ...................................... SKIPPED
    [INFO] Hive HBase Handler ................................ SKIPPED
    [INFO] Hive HCatalog ..................................... SKIPPED
    [INFO] Hive HCatalog Core ................................ SKIPPED
    [INFO] Hive HCatalog Pig Adapter ......................... SKIPPED
    [INFO] Hive HCatalog Server Extensions ................... SKIPPED
    [INFO] Hive HCatalog Webhcat Java Client ................. SKIPPED
    [INFO] Hive HCatalog Webhcat ............................. SKIPPED
    [INFO] Hive HCatalog Streaming ........................... SKIPPED
    [INFO] Hive HWI .......................................... SKIPPED
    [INFO] Hive ODBC ......................................... SKIPPED
    [INFO] Hive Shims Aggregator ............................. SKIPPED
    [INFO] Hive TestUtils .................................... SKIPPED
    [INFO] Hive Packaging .................................... SKIPPED
    [INFO] ------------------------------------------------------------------------
    [INFO] BUILD FAILURE
    [INFO] ------------------------------------------------------------------------
    [INFO] Total time: 2:22.968s
    [INFO] Finished at: Wed Oct 08 13:40:13 EDT 2014
    [INFO] Final Memory: 77M/417M
    [INFO] ------------------------------------------------------------------------
    [ERROR] Failed to execute goal org.apache.maven.plugins:maven-remote-resources-plugin:1.5:process (default) on project hive-exec: Error resolving project artifact: Could not transfer artifact net.hydromatic:linq4j:pom:0.4 from/to datanucleus (http://www.datanucleus.org/downloads/maven2): Access denied to: http://www.datanucleus.org/downloads/maven2/net/hydromatic/linq4j/0.4/linq4j-0.4.pom, ReasonPhrase: Forbidden. for project net.hydromatic:linq4j:jar:0.4 -> [Help 1]
    [ERROR]
    [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
    [ERROR] Re-run Maven using the -X switch to enable full debug logging.
    [ERROR]
    [ERROR] For more information about the errors and possible solutions, please read the following articles:
    [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
    [ERROR]
    [ERROR] After correcting the problems, you can resume the build with the command
    [ERROR] mvn <goals> -rf :hive-exec
    + exit 1
    '
    {noformat}

    This message is automatically generated.

    ATTACHMENT ID: 12673527
    CBO trunk merge: union11 test fails due to incorrect plan
    ---------------------------------------------------------

    Key: HIVE-8225
    URL: https://issues.apache.org/jira/browse/HIVE-8225
    Project: Hive
    Issue Type: Bug
    Reporter: Sergey Shelukhin
    Assignee: Sergey Shelukhin
    Priority: Critical
    Fix For: 0.14.0

    Attachments: HIVE-8225.1.patch, HIVE-8225.2.patch, HIVE-8225.3.patch, HIVE-8225.4.patch, HIVE-8225.5.patch, HIVE-8225.6.patch, HIVE-8225.7.patch, HIVE-8225.inprogress.patch, HIVE-8225.inprogress.patch, HIVE-8225.patch


    The result changes to as if the union didn't have count() inside. The issue can be fixed by using srcunion.value outside the subquery in count (replace count(1) with count(srcunion.value)). Otherwise, it looks like count(1) node from union-ed queries is not present in AST at all, which might cause this result.
    -Interestingly, adding group by to each query in a union produces completely weird result (count(1) is 309 for each key, whereas it should be 1 and the "logical" incorrect value if internal count is lost is 500)- Nm, that groups by table column called key, which is weird but is what Hive does


    --
    This message was sent by Atlassian JIRA
    (v6.3.4#6332)
  • Sergey Shelukhin (JIRA) at Oct 8, 2014 at 10:47 pm
    [ https://issues.apache.org/jira/browse/HIVE-8225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14164316#comment-14164316 ]

    Sergey Shelukhin commented on HIVE-8225:
    ----------------------------------------

    is it possible to resubmit patch for HiveQA?

    CBO trunk merge: union11 test fails due to incorrect plan
    ---------------------------------------------------------

    Key: HIVE-8225
    URL: https://issues.apache.org/jira/browse/HIVE-8225
    Project: Hive
    Issue Type: Bug
    Reporter: Sergey Shelukhin
    Assignee: Sergey Shelukhin
    Priority: Critical
    Fix For: 0.14.0

    Attachments: HIVE-8225.1.patch, HIVE-8225.2.patch, HIVE-8225.3.patch, HIVE-8225.4.patch, HIVE-8225.5.patch, HIVE-8225.6.patch, HIVE-8225.7.patch, HIVE-8225.inprogress.patch, HIVE-8225.inprogress.patch, HIVE-8225.patch


    The result changes to as if the union didn't have count() inside. The issue can be fixed by using srcunion.value outside the subquery in count (replace count(1) with count(srcunion.value)). Otherwise, it looks like count(1) node from union-ed queries is not present in AST at all, which might cause this result.
    -Interestingly, adding group by to each query in a union produces completely weird result (count(1) is 309 for each key, whereas it should be 1 and the "logical" incorrect value if internal count is lost is 500)- Nm, that groups by table column called key, which is weird but is what Hive does


    --
    This message was sent by Atlassian JIRA
    (v6.3.4#6332)
  • Sergey Shelukhin (JIRA) at Oct 8, 2014 at 10:47 pm
    [ https://issues.apache.org/jira/browse/HIVE-8225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Sergey Shelukhin updated HIVE-8225:
    -----------------------------------
         Assignee: Pengcheng Xiong (was: Sergey Shelukhin)
    CBO trunk merge: union11 test fails due to incorrect plan
    ---------------------------------------------------------

    Key: HIVE-8225
    URL: https://issues.apache.org/jira/browse/HIVE-8225
    Project: Hive
    Issue Type: Bug
    Reporter: Sergey Shelukhin
    Assignee: Pengcheng Xiong
    Priority: Critical
    Fix For: 0.14.0

    Attachments: HIVE-8225.1.patch, HIVE-8225.2.patch, HIVE-8225.3.patch, HIVE-8225.4.patch, HIVE-8225.5.patch, HIVE-8225.6.patch, HIVE-8225.7.patch, HIVE-8225.inprogress.patch, HIVE-8225.inprogress.patch, HIVE-8225.patch


    The result changes to as if the union didn't have count() inside. The issue can be fixed by using srcunion.value outside the subquery in count (replace count(1) with count(srcunion.value)). Otherwise, it looks like count(1) node from union-ed queries is not present in AST at all, which might cause this result.
    -Interestingly, adding group by to each query in a union produces completely weird result (count(1) is 309 for each key, whereas it should be 1 and the "logical" incorrect value if internal count is lost is 500)- Nm, that groups by table column called key, which is weird but is what Hive does


    --
    This message was sent by Atlassian JIRA
    (v6.3.4#6332)
  • Pengcheng Xiong (JIRA) at Oct 8, 2014 at 10:50 pm
    [ https://issues.apache.org/jira/browse/HIVE-8225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Pengcheng Xiong updated HIVE-8225:
    ----------------------------------
         Status: Open (was: Patch Available)
    CBO trunk merge: union11 test fails due to incorrect plan
    ---------------------------------------------------------

    Key: HIVE-8225
    URL: https://issues.apache.org/jira/browse/HIVE-8225
    Project: Hive
    Issue Type: Bug
    Reporter: Sergey Shelukhin
    Assignee: Pengcheng Xiong
    Priority: Critical
    Fix For: 0.14.0

    Attachments: HIVE-8225.1.patch, HIVE-8225.2.patch, HIVE-8225.3.patch, HIVE-8225.4.patch, HIVE-8225.5.patch, HIVE-8225.6.patch, HIVE-8225.7.patch, HIVE-8225.inprogress.patch, HIVE-8225.inprogress.patch, HIVE-8225.patch


    The result changes to as if the union didn't have count() inside. The issue can be fixed by using srcunion.value outside the subquery in count (replace count(1) with count(srcunion.value)). Otherwise, it looks like count(1) node from union-ed queries is not present in AST at all, which might cause this result.
    -Interestingly, adding group by to each query in a union produces completely weird result (count(1) is 309 for each key, whereas it should be 1 and the "logical" incorrect value if internal count is lost is 500)- Nm, that groups by table column called key, which is weird but is what Hive does


    --
    This message was sent by Atlassian JIRA
    (v6.3.4#6332)
  • Pengcheng Xiong (JIRA) at Oct 8, 2014 at 10:51 pm
    [ https://issues.apache.org/jira/browse/HIVE-8225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Pengcheng Xiong updated HIVE-8225:
    ----------------------------------
         Attachment: HIVE-8225.7.patch

    resubmit patch
    CBO trunk merge: union11 test fails due to incorrect plan
    ---------------------------------------------------------

    Key: HIVE-8225
    URL: https://issues.apache.org/jira/browse/HIVE-8225
    Project: Hive
    Issue Type: Bug
    Reporter: Sergey Shelukhin
    Assignee: Pengcheng Xiong
    Priority: Critical
    Fix For: 0.14.0

    Attachments: HIVE-8225.1.patch, HIVE-8225.2.patch, HIVE-8225.3.patch, HIVE-8225.4.patch, HIVE-8225.5.patch, HIVE-8225.6.patch, HIVE-8225.7.patch, HIVE-8225.7.patch, HIVE-8225.inprogress.patch, HIVE-8225.inprogress.patch, HIVE-8225.patch


    The result changes to as if the union didn't have count() inside. The issue can be fixed by using srcunion.value outside the subquery in count (replace count(1) with count(srcunion.value)). Otherwise, it looks like count(1) node from union-ed queries is not present in AST at all, which might cause this result.
    -Interestingly, adding group by to each query in a union produces completely weird result (count(1) is 309 for each key, whereas it should be 1 and the "logical" incorrect value if internal count is lost is 500)- Nm, that groups by table column called key, which is weird but is what Hive does


    --
    This message was sent by Atlassian JIRA
    (v6.3.4#6332)
  • Pengcheng Xiong (JIRA) at Oct 9, 2014 at 8:14 pm
    [ https://issues.apache.org/jira/browse/HIVE-8225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Pengcheng Xiong updated HIVE-8225:
    ----------------------------------
         Status: Patch Available (was: Open)
    CBO trunk merge: union11 test fails due to incorrect plan
    ---------------------------------------------------------

    Key: HIVE-8225
    URL: https://issues.apache.org/jira/browse/HIVE-8225
    Project: Hive
    Issue Type: Bug
    Reporter: Sergey Shelukhin
    Assignee: Pengcheng Xiong
    Priority: Critical
    Fix For: 0.14.0

    Attachments: HIVE-8225.1.patch, HIVE-8225.2.patch, HIVE-8225.3.patch, HIVE-8225.4.patch, HIVE-8225.5.patch, HIVE-8225.6.patch, HIVE-8225.7.patch, HIVE-8225.inprogress.patch, HIVE-8225.inprogress.patch, HIVE-8225.patch


    The result changes to as if the union didn't have count() inside. The issue can be fixed by using srcunion.value outside the subquery in count (replace count(1) with count(srcunion.value)). Otherwise, it looks like count(1) node from union-ed queries is not present in AST at all, which might cause this result.
    -Interestingly, adding group by to each query in a union produces completely weird result (count(1) is 309 for each key, whereas it should be 1 and the "logical" incorrect value if internal count is lost is 500)- Nm, that groups by table column called key, which is weird but is what Hive does


    --
    This message was sent by Atlassian JIRA
    (v6.3.4#6332)
  • Pengcheng Xiong (JIRA) at Oct 9, 2014 at 8:14 pm
    [ https://issues.apache.org/jira/browse/HIVE-8225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Pengcheng Xiong updated HIVE-8225:
    ----------------------------------
         Attachment: (was: HIVE-8225.7.patch)
    CBO trunk merge: union11 test fails due to incorrect plan
    ---------------------------------------------------------

    Key: HIVE-8225
    URL: https://issues.apache.org/jira/browse/HIVE-8225
    Project: Hive
    Issue Type: Bug
    Reporter: Sergey Shelukhin
    Assignee: Pengcheng Xiong
    Priority: Critical
    Fix For: 0.14.0

    Attachments: HIVE-8225.1.patch, HIVE-8225.2.patch, HIVE-8225.3.patch, HIVE-8225.4.patch, HIVE-8225.5.patch, HIVE-8225.6.patch, HIVE-8225.7.patch, HIVE-8225.inprogress.patch, HIVE-8225.inprogress.patch, HIVE-8225.patch


    The result changes to as if the union didn't have count() inside. The issue can be fixed by using srcunion.value outside the subquery in count (replace count(1) with count(srcunion.value)). Otherwise, it looks like count(1) node from union-ed queries is not present in AST at all, which might cause this result.
    -Interestingly, adding group by to each query in a union produces completely weird result (count(1) is 309 for each key, whereas it should be 1 and the "logical" incorrect value if internal count is lost is 500)- Nm, that groups by table column called key, which is weird but is what Hive does


    --
    This message was sent by Atlassian JIRA
    (v6.3.4#6332)
  • Pengcheng Xiong (JIRA) at Oct 11, 2014 at 12:13 am
    [ https://issues.apache.org/jira/browse/HIVE-8225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Pengcheng Xiong updated HIVE-8225:
    ----------------------------------
         Attachment: HIVE-8225.8.patch

    resubmit patch
    CBO trunk merge: union11 test fails due to incorrect plan
    ---------------------------------------------------------

    Key: HIVE-8225
    URL: https://issues.apache.org/jira/browse/HIVE-8225
    Project: Hive
    Issue Type: Bug
    Reporter: Sergey Shelukhin
    Assignee: Pengcheng Xiong
    Priority: Critical
    Fix For: 0.14.0

    Attachments: HIVE-8225.1.patch, HIVE-8225.2.patch, HIVE-8225.3.patch, HIVE-8225.4.patch, HIVE-8225.5.patch, HIVE-8225.6.patch, HIVE-8225.7.patch, HIVE-8225.8.patch, HIVE-8225.inprogress.patch, HIVE-8225.inprogress.patch, HIVE-8225.patch


    The result changes to as if the union didn't have count() inside. The issue can be fixed by using srcunion.value outside the subquery in count (replace count(1) with count(srcunion.value)). Otherwise, it looks like count(1) node from union-ed queries is not present in AST at all, which might cause this result.
    -Interestingly, adding group by to each query in a union produces completely weird result (count(1) is 309 for each key, whereas it should be 1 and the "logical" incorrect value if internal count is lost is 500)- Nm, that groups by table column called key, which is weird but is what Hive does


    --
    This message was sent by Atlassian JIRA
    (v6.3.4#6332)
  • Pengcheng Xiong (JIRA) at Oct 11, 2014 at 12:13 am
    [ https://issues.apache.org/jira/browse/HIVE-8225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Pengcheng Xiong updated HIVE-8225:
    ----------------------------------
         Status: Patch Available (was: Open)
    CBO trunk merge: union11 test fails due to incorrect plan
    ---------------------------------------------------------

    Key: HIVE-8225
    URL: https://issues.apache.org/jira/browse/HIVE-8225
    Project: Hive
    Issue Type: Bug
    Reporter: Sergey Shelukhin
    Assignee: Pengcheng Xiong
    Priority: Critical
    Fix For: 0.14.0

    Attachments: HIVE-8225.1.patch, HIVE-8225.2.patch, HIVE-8225.3.patch, HIVE-8225.4.patch, HIVE-8225.5.patch, HIVE-8225.6.patch, HIVE-8225.7.patch, HIVE-8225.8.patch, HIVE-8225.inprogress.patch, HIVE-8225.inprogress.patch, HIVE-8225.patch


    The result changes to as if the union didn't have count() inside. The issue can be fixed by using srcunion.value outside the subquery in count (replace count(1) with count(srcunion.value)). Otherwise, it looks like count(1) node from union-ed queries is not present in AST at all, which might cause this result.
    -Interestingly, adding group by to each query in a union produces completely weird result (count(1) is 309 for each key, whereas it should be 1 and the "logical" incorrect value if internal count is lost is 500)- Nm, that groups by table column called key, which is weird but is what Hive does


    --
    This message was sent by Atlassian JIRA
    (v6.3.4#6332)
  • Pengcheng Xiong (JIRA) at Oct 11, 2014 at 12:13 am
    [ https://issues.apache.org/jira/browse/HIVE-8225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Pengcheng Xiong updated HIVE-8225:
    ----------------------------------
         Status: Open (was: Patch Available)
    CBO trunk merge: union11 test fails due to incorrect plan
    ---------------------------------------------------------

    Key: HIVE-8225
    URL: https://issues.apache.org/jira/browse/HIVE-8225
    Project: Hive
    Issue Type: Bug
    Reporter: Sergey Shelukhin
    Assignee: Pengcheng Xiong
    Priority: Critical
    Fix For: 0.14.0

    Attachments: HIVE-8225.1.patch, HIVE-8225.2.patch, HIVE-8225.3.patch, HIVE-8225.4.patch, HIVE-8225.5.patch, HIVE-8225.6.patch, HIVE-8225.7.patch, HIVE-8225.8.patch, HIVE-8225.inprogress.patch, HIVE-8225.inprogress.patch, HIVE-8225.patch


    The result changes to as if the union didn't have count() inside. The issue can be fixed by using srcunion.value outside the subquery in count (replace count(1) with count(srcunion.value)). Otherwise, it looks like count(1) node from union-ed queries is not present in AST at all, which might cause this result.
    -Interestingly, adding group by to each query in a union produces completely weird result (count(1) is 309 for each key, whereas it should be 1 and the "logical" incorrect value if internal count is lost is 500)- Nm, that groups by table column called key, which is weird but is what Hive does


    --
    This message was sent by Atlassian JIRA
    (v6.3.4#6332)
  • Hive QA (JIRA) at Oct 11, 2014 at 1:01 pm
    [ https://issues.apache.org/jira/browse/HIVE-8225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14168135#comment-14168135 ]

    Hive QA commented on HIVE-8225:
    -------------------------------



    {color:red}Overall{color}: -1 at least one tests failed

    Here are the results of testing the latest attachment:
    https://issues.apache.org/jira/secure/attachment/12674310/HIVE-8225.8.patch

    {color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 4137 tests executed
    *Failed tests:*
    {noformat}
    org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_tez_smb_1
    {noformat}

    Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/1212/testReport
    Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/1212/console
    Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-1212/

    Messages:
    {noformat}
    Executing org.apache.hive.ptest.execution.PrepPhase
    Executing org.apache.hive.ptest.execution.ExecutionPhase
    Executing org.apache.hive.ptest.execution.ReportingPhase
    Tests exited with: TestsFailedException: 1 tests failed
    {noformat}

    This message is automatically generated.

    ATTACHMENT ID: 12674310
    CBO trunk merge: union11 test fails due to incorrect plan
    ---------------------------------------------------------

    Key: HIVE-8225
    URL: https://issues.apache.org/jira/browse/HIVE-8225
    Project: Hive
    Issue Type: Bug
    Reporter: Sergey Shelukhin
    Assignee: Pengcheng Xiong
    Priority: Critical
    Fix For: 0.14.0

    Attachments: HIVE-8225.1.patch, HIVE-8225.2.patch, HIVE-8225.3.patch, HIVE-8225.4.patch, HIVE-8225.5.patch, HIVE-8225.6.patch, HIVE-8225.7.patch, HIVE-8225.8.patch, HIVE-8225.inprogress.patch, HIVE-8225.inprogress.patch, HIVE-8225.patch


    The result changes to as if the union didn't have count() inside. The issue can be fixed by using srcunion.value outside the subquery in count (replace count(1) with count(srcunion.value)). Otherwise, it looks like count(1) node from union-ed queries is not present in AST at all, which might cause this result.
    -Interestingly, adding group by to each query in a union produces completely weird result (count(1) is 309 for each key, whereas it should be 1 and the "logical" incorrect value if internal count is lost is 500)- Nm, that groups by table column called key, which is weird but is what Hive does


    --
    This message was sent by Atlassian JIRA
    (v6.3.4#6332)
  • Pengcheng Xiong (JIRA) at Oct 11, 2014 at 9:25 pm
    [ https://issues.apache.org/jira/browse/HIVE-8225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14168348#comment-14168348 ]

    Pengcheng Xiong commented on HIVE-8225:
    ---------------------------------------

    Dear QA, I got [INFO] BUILD SUCCESS for the failed test testCliDriver_tez_smb_1.q
    CBO trunk merge: union11 test fails due to incorrect plan
    ---------------------------------------------------------

    Key: HIVE-8225
    URL: https://issues.apache.org/jira/browse/HIVE-8225
    Project: Hive
    Issue Type: Bug
    Reporter: Sergey Shelukhin
    Assignee: Pengcheng Xiong
    Priority: Critical
    Fix For: 0.14.0

    Attachments: HIVE-8225.1.patch, HIVE-8225.2.patch, HIVE-8225.3.patch, HIVE-8225.4.patch, HIVE-8225.5.patch, HIVE-8225.6.patch, HIVE-8225.7.patch, HIVE-8225.8.patch, HIVE-8225.inprogress.patch, HIVE-8225.inprogress.patch, HIVE-8225.patch


    The result changes to as if the union didn't have count() inside. The issue can be fixed by using srcunion.value outside the subquery in count (replace count(1) with count(srcunion.value)). Otherwise, it looks like count(1) node from union-ed queries is not present in AST at all, which might cause this result.
    -Interestingly, adding group by to each query in a union produces completely weird result (count(1) is 309 for each key, whereas it should be 1 and the "logical" incorrect value if internal count is lost is 500)- Nm, that groups by table column called key, which is weird but is what Hive does


    --
    This message was sent by Atlassian JIRA
    (v6.3.4#6332)
  • Sergey Shelukhin (JIRA) at Oct 13, 2014 at 9:35 pm
    [ https://issues.apache.org/jira/browse/HIVE-8225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14170031#comment-14170031 ]

    Sergey Shelukhin commented on HIVE-8225:
    ----------------------------------------

    +1 :)
    CBO trunk merge: union11 test fails due to incorrect plan
    ---------------------------------------------------------

    Key: HIVE-8225
    URL: https://issues.apache.org/jira/browse/HIVE-8225
    Project: Hive
    Issue Type: Bug
    Reporter: Sergey Shelukhin
    Assignee: Pengcheng Xiong
    Priority: Critical
    Fix For: 0.14.0

    Attachments: HIVE-8225.1.patch, HIVE-8225.2.patch, HIVE-8225.3.patch, HIVE-8225.4.patch, HIVE-8225.5.patch, HIVE-8225.6.patch, HIVE-8225.7.patch, HIVE-8225.8.patch, HIVE-8225.inprogress.patch, HIVE-8225.inprogress.patch, HIVE-8225.patch


    The result changes to as if the union didn't have count() inside. The issue can be fixed by using srcunion.value outside the subquery in count (replace count(1) with count(srcunion.value)). Otherwise, it looks like count(1) node from union-ed queries is not present in AST at all, which might cause this result.
    -Interestingly, adding group by to each query in a union produces completely weird result (count(1) is 309 for each key, whereas it should be 1 and the "logical" incorrect value if internal count is lost is 500)- Nm, that groups by table column called key, which is weird but is what Hive does


    --
    This message was sent by Atlassian JIRA
    (v6.3.4#6332)
  • Sergey Shelukhin (JIRA) at Oct 13, 2014 at 10:14 pm
    [ https://issues.apache.org/jira/browse/HIVE-8225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Sergey Shelukhin updated HIVE-8225:
    -----------------------------------
            Resolution: Fixed
         Fix Version/s: (was: 0.14.0)
                        0.15.0
                Status: Resolved (was: Patch Available)

    committed to trunk. [~vikram.dixit] ok for 14?
    CBO trunk merge: union11 test fails due to incorrect plan
    ---------------------------------------------------------

    Key: HIVE-8225
    URL: https://issues.apache.org/jira/browse/HIVE-8225
    Project: Hive
    Issue Type: Bug
    Reporter: Sergey Shelukhin
    Assignee: Pengcheng Xiong
    Priority: Critical
    Fix For: 0.15.0

    Attachments: HIVE-8225.1.patch, HIVE-8225.2.patch, HIVE-8225.3.patch, HIVE-8225.4.patch, HIVE-8225.5.patch, HIVE-8225.6.patch, HIVE-8225.7.patch, HIVE-8225.8.patch, HIVE-8225.inprogress.patch, HIVE-8225.inprogress.patch, HIVE-8225.patch


    The result changes to as if the union didn't have count() inside. The issue can be fixed by using srcunion.value outside the subquery in count (replace count(1) with count(srcunion.value)). Otherwise, it looks like count(1) node from union-ed queries is not present in AST at all, which might cause this result.
    -Interestingly, adding group by to each query in a union produces completely weird result (count(1) is 309 for each key, whereas it should be 1 and the "logical" incorrect value if internal count is lost is 500)- Nm, that groups by table column called key, which is weird but is what Hive does


    --
    This message was sent by Atlassian JIRA
    (v6.3.4#6332)
  • Vikram Dixit K (JIRA) at Oct 14, 2014 at 12:38 am
    [ https://issues.apache.org/jira/browse/HIVE-8225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14170300#comment-14170300 ]

    Vikram Dixit K commented on HIVE-8225:
    --------------------------------------

    +1 for 0.14
    CBO trunk merge: union11 test fails due to incorrect plan
    ---------------------------------------------------------

    Key: HIVE-8225
    URL: https://issues.apache.org/jira/browse/HIVE-8225
    Project: Hive
    Issue Type: Bug
    Reporter: Sergey Shelukhin
    Assignee: Pengcheng Xiong
    Priority: Critical
    Fix For: 0.15.0

    Attachments: HIVE-8225.1.patch, HIVE-8225.2.patch, HIVE-8225.3.patch, HIVE-8225.4.patch, HIVE-8225.5.patch, HIVE-8225.6.patch, HIVE-8225.7.patch, HIVE-8225.8.patch, HIVE-8225.inprogress.patch, HIVE-8225.inprogress.patch, HIVE-8225.patch


    The result changes to as if the union didn't have count() inside. The issue can be fixed by using srcunion.value outside the subquery in count (replace count(1) with count(srcunion.value)). Otherwise, it looks like count(1) node from union-ed queries is not present in AST at all, which might cause this result.
    -Interestingly, adding group by to each query in a union produces completely weird result (count(1) is 309 for each key, whereas it should be 1 and the "logical" incorrect value if internal count is lost is 500)- Nm, that groups by table column called key, which is weird but is what Hive does


    --
    This message was sent by Atlassian JIRA
    (v6.3.4#6332)
  • Sergey Shelukhin (JIRA) at Oct 14, 2014 at 10:54 pm
    [ https://issues.apache.org/jira/browse/HIVE-8225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Sergey Shelukhin updated HIVE-8225:
    -----------------------------------
         Fix Version/s: (was: 0.15.0)
                        0.14.0
    CBO trunk merge: union11 test fails due to incorrect plan
    ---------------------------------------------------------

    Key: HIVE-8225
    URL: https://issues.apache.org/jira/browse/HIVE-8225
    Project: Hive
    Issue Type: Bug
    Reporter: Sergey Shelukhin
    Assignee: Pengcheng Xiong
    Priority: Critical
    Fix For: 0.14.0

    Attachments: HIVE-8225.1.patch, HIVE-8225.2.patch, HIVE-8225.3.patch, HIVE-8225.4.patch, HIVE-8225.5.patch, HIVE-8225.6.patch, HIVE-8225.7.patch, HIVE-8225.8.patch, HIVE-8225.inprogress.patch, HIVE-8225.inprogress.patch, HIVE-8225.patch


    The result changes to as if the union didn't have count() inside. The issue can be fixed by using srcunion.value outside the subquery in count (replace count(1) with count(srcunion.value)). Otherwise, it looks like count(1) node from union-ed queries is not present in AST at all, which might cause this result.
    -Interestingly, adding group by to each query in a union produces completely weird result (count(1) is 309 for each key, whereas it should be 1 and the "logical" incorrect value if internal count is lost is 500)- Nm, that groups by table column called key, which is weird but is what Hive does


    --
    This message was sent by Atlassian JIRA
    (v6.3.4#6332)
  • Sergey Shelukhin (JIRA) at Oct 14, 2014 at 10:55 pm
    [ https://issues.apache.org/jira/browse/HIVE-8225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14171672#comment-14171672 ]

    Sergey Shelukhin commented on HIVE-8225:
    ----------------------------------------

    committed to 14
    CBO trunk merge: union11 test fails due to incorrect plan
    ---------------------------------------------------------

    Key: HIVE-8225
    URL: https://issues.apache.org/jira/browse/HIVE-8225
    Project: Hive
    Issue Type: Bug
    Reporter: Sergey Shelukhin
    Assignee: Pengcheng Xiong
    Priority: Critical
    Fix For: 0.14.0

    Attachments: HIVE-8225.1.patch, HIVE-8225.2.patch, HIVE-8225.3.patch, HIVE-8225.4.patch, HIVE-8225.5.patch, HIVE-8225.6.patch, HIVE-8225.7.patch, HIVE-8225.8.patch, HIVE-8225.inprogress.patch, HIVE-8225.inprogress.patch, HIVE-8225.patch


    The result changes to as if the union didn't have count() inside. The issue can be fixed by using srcunion.value outside the subquery in count (replace count(1) with count(srcunion.value)). Otherwise, it looks like count(1) node from union-ed queries is not present in AST at all, which might cause this result.
    -Interestingly, adding group by to each query in a union produces completely weird result (count(1) is 309 for each key, whereas it should be 1 and the "logical" incorrect value if internal count is lost is 500)- Nm, that groups by table column called key, which is weird but is what Hive does


    --
    This message was sent by Atlassian JIRA
    (v6.3.4#6332)

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupdev @
categorieshive, hadoop
postedOct 1, '14 at 7:28a
activeOct 14, '14 at 10:55p
posts48
users1
websitehive.apache.org

1 user in discussion

Sergey Shelukhin (JIRA): 48 posts

People

Translate

site design / logo © 2021 Grokbase