FAQ
Repository: hive
Updated Branches:
   refs/heads/branch-1 af07b4171 -> 234972142


HIVE-12189: The list in pushdownPreds of ppd.ExprWalkerInfo should not be allowed to grow very large (Yongzhi Chen, reviewed by Chao Sun)


Project: http://git-wip-us.apache.org/repos/asf/hive/repo
Commit: http://git-wip-us.apache.org/repos/asf/hive/commit/23497214
Tree: http://git-wip-us.apache.org/repos/asf/hive/tree/23497214
Diff: http://git-wip-us.apache.org/repos/asf/hive/diff/23497214

Branch: refs/heads/branch-1
Commit: 2349721424a6ca014d4dae95d2c326c54a771fe6
Parents: af07b41
Author: Yongzhi Chen <yongzhi_chen@hotmail.com>
Authored: Sun Oct 25 23:25:12 2015 -0700
Committer: Chao Sun <sunchao@apache.org>
Committed: Sun Oct 25 23:25:12 2015 -0700

----------------------------------------------------------------------
  .../hadoop/hive/ql/ppd/ExprWalkerInfo.java | 23 ++++++++++++++++++--
  1 file changed, 21 insertions(+), 2 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/hive/blob/23497214/ql/src/java/org/apache/hadoop/hive/ql/ppd/ExprWalkerInfo.java
----------------------------------------------------------------------
diff --git a/ql/src/java/org/apache/hadoop/hive/ql/ppd/ExprWalkerInfo.java b/ql/src/java/org/apache/hadoop/hive/ql/ppd/ExprWalkerInfo.java
index e4b768e..fca671c 100644
--- a/ql/src/java/org/apache/hadoop/hive/ql/ppd/ExprWalkerInfo.java
+++ b/ql/src/java/org/apache/hadoop/hive/ql/ppd/ExprWalkerInfo.java
@@ -149,7 +149,13 @@ public class ExprWalkerInfo implements NodeProcessorCtx {
    }

    public void addFinalCandidate(String alias, ExprNodeDesc expr) {
- getPushdownPreds(alias).add(expr);
+ List<ExprNodeDesc> predicates = getPushdownPreds(alias);
+ for (ExprNodeDesc curPred: predicates) {
+ if (curPred.isSame(expr)) {
+ return;
+ }
+ }
+ predicates.add(expr);
    }

    /**
@@ -159,7 +165,20 @@ public class ExprWalkerInfo implements NodeProcessorCtx {
     * @param pushDowns
     */
    public void addPushDowns(String alias, List<ExprNodeDesc> pushDowns) {
- getPushdownPreds(alias).addAll(pushDowns);
+ List<ExprNodeDesc> predicates = getPushdownPreds(alias);
+ boolean isNew;
+ for (ExprNodeDesc newPred: pushDowns) {
+ isNew = true;
+ for (ExprNodeDesc curPred: predicates) {
+ if (curPred.isSame(newPred)) {
+ isNew = false;
+ break;
+ }
+ }
+ if (isNew) {
+ predicates.add(newPred);
+ }
+ }
    }

    /**

Search Discussions

  • Sunchao at Oct 26, 2015 at 6:25 am
    Repository: hive
    Updated Branches:
       refs/heads/master 9ea51d122 -> f415ce956


    HIVE-12189: The list in pushdownPreds of ppd.ExprWalkerInfo should not be allowed to grow very large (Yongzhi Chen, reviewed by Chao Sun)


    Project: http://git-wip-us.apache.org/repos/asf/hive/repo
    Commit: http://git-wip-us.apache.org/repos/asf/hive/commit/f415ce95
    Tree: http://git-wip-us.apache.org/repos/asf/hive/tree/f415ce95
    Diff: http://git-wip-us.apache.org/repos/asf/hive/diff/f415ce95

    Branch: refs/heads/master
    Commit: f415ce9567661dcdc4b68b49ccb6ee4962a330e4
    Parents: 9ea51d1
    Author: Yongzhi Chen <yongzhi_chen@hotmail.com>
    Authored: Sun Oct 25 23:24:31 2015 -0700
    Committer: Chao Sun <sunchao@apache.org>
    Committed: Sun Oct 25 23:24:31 2015 -0700

    ----------------------------------------------------------------------
      .../hadoop/hive/ql/ppd/ExprWalkerInfo.java | 23 ++++++++++++++++++--
      1 file changed, 21 insertions(+), 2 deletions(-)
    ----------------------------------------------------------------------


    http://git-wip-us.apache.org/repos/asf/hive/blob/f415ce95/ql/src/java/org/apache/hadoop/hive/ql/ppd/ExprWalkerInfo.java
    ----------------------------------------------------------------------
    diff --git a/ql/src/java/org/apache/hadoop/hive/ql/ppd/ExprWalkerInfo.java b/ql/src/java/org/apache/hadoop/hive/ql/ppd/ExprWalkerInfo.java
    index e4b768e..fca671c 100644
    --- a/ql/src/java/org/apache/hadoop/hive/ql/ppd/ExprWalkerInfo.java
    +++ b/ql/src/java/org/apache/hadoop/hive/ql/ppd/ExprWalkerInfo.java
    @@ -149,7 +149,13 @@ public class ExprWalkerInfo implements NodeProcessorCtx {
        }

        public void addFinalCandidate(String alias, ExprNodeDesc expr) {
    - getPushdownPreds(alias).add(expr);
    + List<ExprNodeDesc> predicates = getPushdownPreds(alias);
    + for (ExprNodeDesc curPred: predicates) {
    + if (curPred.isSame(expr)) {
    + return;
    + }
    + }
    + predicates.add(expr);
        }

        /**
    @@ -159,7 +165,20 @@ public class ExprWalkerInfo implements NodeProcessorCtx {
         * @param pushDowns
         */
        public void addPushDowns(String alias, List<ExprNodeDesc> pushDowns) {
    - getPushdownPreds(alias).addAll(pushDowns);
    + List<ExprNodeDesc> predicates = getPushdownPreds(alias);
    + boolean isNew;
    + for (ExprNodeDesc newPred: pushDowns) {
    + isNew = true;
    + for (ExprNodeDesc curPred: predicates) {
    + if (curPred.isSame(newPred)) {
    + isNew = false;
    + break;
    + }
    + }
    + if (isNew) {
    + predicates.add(newPred);
    + }
    + }
        }

        /**

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommits @
categorieshive, hadoop
postedOct 26, '15 at 6:25a
activeOct 26, '15 at 6:25a
posts2
users1
websitehive.apache.org

1 user in discussion

Sunchao: 2 posts

People

Translate

site design / logo © 2021 Grokbase