Grokbase Groups HBase user June 2016
FAQ
Which version of hbase / Hadoop are you using ?
(So that line number matching can be more accurate)

It would be good if you can show your code snippet.

Thanks
On Jun 11, 2016, at 12:57 AM, Jilani Shaik wrote:

Hi,

I am trying to do hbase table bulk load from data file using map reduce. in the process I am getting the Null pointer exception during the shuffle and sort. I have both mapper and reducer in the flow. where as mapper is ending with success and reducer not being invoked even setup.

It is showing the below error in terminal where the job executed.

Error: java.lang.NullPointerException
at com.google.common.base.Preconditions.checkNotNull(Preconditions.java:191)
at com.google.common.base.Splitter.split(Splitter.java:375)
at org.apache.hadoop.hbase.mapreduce.ImportTsv$TsvParser.<init>(ImportTsv.java:151)
at org.apache.hadoop.hbase.mapreduce.TextSortReducer.setup(TextSortReducer.java:105)
at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:168)
at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:627)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:389)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1566)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)


Please let me know what could be the issue.


Thanks,
Jilani

Search Discussions

  • Jilani Shaik at Jun 15, 2016 at 6:05 pm
    Please let me know what am I missing here.

    I am using MapR hadoop. Please find the classpath which is showing all the
    jars with versions, mean while I will get the code snippet also. I am using
    bulk write approach with mapper and reducer. HBase table created with bulk
    load enabled as true.

    + HADOOP_CLASSPATH='/opt/mapr/hbase/hbase-0.98.9/bin/../
    conf:/opt/mapr/java/jdk1.7.0_25/lib/tools.jar:/opt/mapr/hbase/hbase
    -0.98.9/bin/..:/opt/mapr/lib/zookeeper-3.4.5-mapr-1406.jar:/opt/mapr/hbase/
    hbase-0.98.9/bin/../lib/activation-1.1.jar:/opt/mapr/hbase/hbase
    -0.98.9/bin/../lib/aopalliance-1.0.jar:/opt/mapr/hbase/hbase
    -0.98.9/bin/../lib/apacheds-i18n-2.0.0-M15.jar:/opt/mapr/hbase/hbase-0.98.9/
    bin/../lib/apacheds-kerberos-codec-2.0.0-M15.jar:/opt/mapr/hbase/hbase
    -0.98.9/bin/../lib/api-asn1-api-1.0.0-M20.jar:/opt/mapr/hbase/hbase-0.98.9/
    bin/../lib/api-util-1.0.0-M20.jar:/opt/mapr/hbase/hbase-0.
    98.9/bin/../lib/asm-3.1.jar:/opt/mapr/hbase/hbase-0.98.9/
    bin/../lib/commons-beanutils-1.7.0.jar:/opt/mapr/hbase/hbase
    -0.98.9/bin/../lib/commons-beanutils-core-1.8.0.jar:/opt/mapr/hbase/hbase-0.
    98.9/bin/../lib/commons-cli-1.2.jar:/opt/mapr/hbase/hbase-0.
    98.9/bin/../lib/commons-codec-1.7.jar:/opt/mapr/hbase/hbase-
    0.98.9/bin/../lib/commons-collections-3.2.1.jar:/opt/mapr/hbase/hbase
    -0.98.9/bin/../lib/commons-compress-1.4.1.jar:/opt/mapr/hbase/hbase-0.
    98.9/bin/../lib/commons-configuration-1.6.jar:/opt/mapr/hbase/hbase
    -0.98.9/bin/../lib/commons-daemon-1.0.13.jar:/opt/mapr/hbase/hbase-0.
    98.9/bin/../lib/commons-digester-1.8.jar:/opt/mapr/hbase/hbase
    -0.98.9/bin/../lib/commons-el-1.0.jar:/opt/mapr/hbase/hbase
    -0.98.9/bin/../lib/commons-httpclient-3.1.jar:/opt/mapr/hbase/hbase-0.98.9/
    bin/../lib/commons-io-2.4.jar:/opt/mapr/hbase/hbase-0.98.9/
    bin/../lib/commons-lang-2.6.jar:/opt/mapr/hbase/hbase-0.
    98.9/bin/../lib/commons-logging-1.1.1.jar:/opt/mapr/hbase/hbase
    -0.98.9/bin/../lib/commons-math-2.1.jar:/opt/mapr/hbase/hbase-0.98.9/bin/..
    /lib/commons-math3-3.1.1.jar:/opt/mapr/hbase/hbase-0.98.9/
    bin/../lib/commons-net-3.1.jar:/opt/mapr/hbase/hbase-0.
    98.9/bin/../lib/findbugs-annotations-1.3.9-1.jar:/opt/mapr/hbase/hbase
    -0.98.9/bin/../lib/guava-12.0.1.jar:/opt/mapr/hbase/hbase-0.98.9/bin/..
    /lib/guice-3.0.jar:/opt/mapr/hbase/hbase-0.98.9/bin/../lib/
    guice-servlet-3.0.jar:/opt/mapr/hbase/hbase-0.98.9/bin/..
    /lib/hamcrest-core-1.3.jar:/opt/mapr/hbase/hbase-0.98.9/bin/../lib/hbase
    -annotations-0.98.9-mapr-1503.jar:/opt/mapr/hbase/hbase-0.98.9/bin/../lib/
    hbase-checkstyle-0.98.9-mapr-1503.jar:/opt/mapr/hbase/hbase
    -0.98.9/bin/../lib/hbase-client-0.98.9-mapr-1503.jar:/opt/mapr/hbase/hbase
    -0.98.9/bin/../lib/hbase-common-0.98.9-mapr-1503.jar:/opt/mapr/hbase/hbase
    -0.98.9/bin/../lib/hbase-common-0.98.9-mapr-1503-tests.jar:/opt/mapr/hbase/
    hbase-0.98.9/bin/../lib/hbase-examples-0.98.9-mapr-1503.jar:/opt/mapr/hbase/
    hbase-0.98.9/bin/../lib/hbase-hadoop2-compat-0.98.9-mapr-1503.jar:/opt/mapr/
    hbase/hbase-0.98.9/bin/../lib/hbase-hadoop-compat-0.98.9-mapr-1503.jar:/
    opt/mapr/hbase/hbase-0.98.9/bin/../lib/hbase-it-0.98.9-
    mapr-1503.jar:/opt/mapr/hbase/hbase-0.98.9/bin/../lib/hbase-
    it-0.98.9-mapr-1503-tests.jar:/opt/mapr/hbase/hbase-0.98.9/bin/../lib/hbase
    -prefix-tree-0.98.9-mapr-1503.jar:/opt/mapr/hbase/hbase-0.98.9/bin/../lib/
    hbase-protocol-0.98.9-mapr-1503.jar:/opt/mapr/hbase/hbase-0.98.9/bin/../lib/
    hbase-rest-0.98.9-mapr-1503.jar:/opt/mapr/hbase/hbase-0.98.9/bin/../lib/
    hbase-server-0.98.9-mapr-1503.jar:/opt/mapr/hbase/hbase-0.98.9/bin/../lib/
    hbase-server-0.98.9-mapr-1503-tests.jar:/opt/mapr/hbase/hbase
    -0.98.9/bin/../lib/hbase-shell-0.98.9-mapr-1503.jar:/opt/mapr/hbase/hbase
    -0.98.9/bin/../lib/hbase-testing-util-0.98.9-mapr-1503.jar:/opt/mapr/hbase/
    hbase-0.98.9/bin/../lib/hbase-thrift-0.98.9-mapr-1503.jar:/opt/mapr/hbase/
    hbase-0.98.9/bin/../lib/high-scale-lib-1.1.1.jar:/opt/mapr/hbase/hbase
    -0.98.9/bin/../lib/htrace-core-2.04.jar:/opt/mapr/hbase/hbase-0.98.9/bin/..
    /lib/httpclient-4.2.5.jar:/opt/mapr/hbase/hbase-0.98.9/
    bin/../lib/httpcore-4.1.3.jar:/opt/mapr/hbase/hbase-0.98.9/
    bin/../lib/jackson-core-asl-1.8.8.jar:/opt/mapr/hbase/hbase-
    0.98.9/bin/../lib/jackson-jaxrs-1.8.8.jar:/opt/mapr/hbase/hbase
    -0.98.9/bin/../lib/jackson-mapper-asl-1.8.8.jar:/opt/mapr/hbase/hbase
    -0.98.9/bin/../lib/jackson-xc-1.8.8.jar:/opt/mapr/hbase/hbase-0.
    98.9/bin/../lib/jamon-runtime-2.3.1.jar:/opt/mapr/hbase/hbase
    -0.98.9/bin/../lib/jasper-compiler-5.5.23.jar:/opt/mapr/hbase/hbase-0.98.9/
    bin/../lib/jasper-runtime-5.5.23.jar:/opt/mapr/hbase/hbase-
    0.98.9/bin/../lib/javax.inject-1.jar:/opt/mapr/hbase/hbase
    -0.98.9/bin/../lib/java-xmlbuilder-0.4.jar:/opt/mapr/hbase/hbase
    -0.98.9/bin/../lib/jaxb-api-2.2.2.jar:/opt/mapr/hbase/hbase
    -0.98.9/bin/../lib/jaxb-impl-2.2.3-1.jar:/opt/mapr/hbase/hbase-0.98.9/bin/..
    /lib/jcodings-1.0.8.jar:/opt/mapr/hbase/hbase-0.98.9/bin/..
    /lib/jersey-client-1.9.jar:/opt/mapr/hbase/hbase-0.98.9/
    bin/../lib/jersey-core-1.8.jar:/opt/mapr/hbase/hbase-0.
    98.9/bin/../lib/jersey-guice-1.9.jar:/opt/mapr/hbase/hbase-
    0.98.9/bin/../lib/jersey-json-1.8.jar:/opt/mapr/hbase/hbase-
    0.98.9/bin/../lib/jersey-server-1.8.jar:/opt/mapr/hbase/hbase
    -0.98.9/bin/../lib/jets3t-0.9.0.jar:/opt/mapr/hbase/hbase-0.98.9/bin/../lib/
    jettison-1.3.1.jar:/opt/mapr/hbase/hbase-0.98.9/bin/../lib/
    jetty-6.1.26.jar:/opt/mapr/hbase/hbase-0.98.9/bin/../lib/
    jetty-sslengine-6.1.26.jar:/opt/mapr/hbase/hbase-0.98.9/
    bin/../lib/jetty-util-6.1.26.jar:/opt/mapr/hbase/hbase-0.
    98.9/bin/../lib/joni-2.1.2.jar:/opt/mapr/hbase/hbase-0.
    98.9/bin/../lib/jruby-complete-1.6.8.jar:/opt/mapr/hbase/hbase
    -0.98.9/bin/../lib/jsch-0.1.42.jar:/opt/mapr/hbase/hbase-0.98.9/bin/../lib/
    jsp-2.1-6.1.14.jar:/opt/mapr/hbase/hbase-0.98.9/bin/../lib/
    jsp-api-2.1-6.1.14.jar:/opt/mapr/hbase/hbase-0.98.9/bin/..
    /lib/jsr305-1.3.9.jar:/opt/mapr/hbase/hbase-0.98.9/bin/..
    /lib/junit-4.11.jar:/opt/mapr/hbase/hbase-0.98.9/bin/../lib/
    leveldbjni-all-1.8.jar:/opt/mapr/hbase/hbase-0.98.9/bin/..
    /lib/libthrift-0.9.0.jar:/opt/mapr/hbase/hbase-0.98.9/bin/..
    /lib/log4j-1.2.17.jar:/opt/mapr/hbase/hbase-0.98.9/bin/..
    /lib/metrics-core-2.2.0.jar:/opt/mapr/hbase/hbase-0.98.9/
    bin/../lib/netty-3.6.6.Final.jar:/opt/mapr/hbase/hbase-0.
    98.9/bin/../lib/paranamer-2.3.jar:/opt/mapr/hbase/hbase-0.
    98.9/bin/../lib/servlet-api-2.5-6.1.14.jar:/opt/mapr/hbase/hbase
    -0.98.9/bin/../lib/servlet-api-2.5.jar:/opt/mapr/hbase/hbase
    -0.98.9/bin/../lib/snappy-java-1.0.4.1.jar:/opt/mapr/hbase/hbase
    -0.98.9/bin/../lib/xmlenc-0.52.jar:/opt/mapr/hbase/hbase-0.98.9/bin/..
    /lib/xz-1.0.jar:/opt/mapr/hbase/hbase-0.98.9/bin/../lib/
    zookeeper-3.4.5-mapr-1406.jar:/opt/mapr/lib/libprotodefs-4.
    0.2-mapr.jar:/opt/mapr/lib/baseutils-4.0.2-mapr.jar:/opt/
    mapr/lib/JPam-1.1.jar:/opt/mapr/hadoop/hadoop-2.5.1/etc/
    hadoop:/opt/mapr/hadoop/hadoop-2.5.1/share/hadoop/
    common/lib/*:/opt/mapr/hadoop/hadoop-2.5.1/share/hadoop/
    common/*:/opt/mapr/hadoop/hadoop-2.5.1/share/hadoop/
    hdfs:/opt/mapr/hadoop/hadoop-2.5.1/share/hadoop/hdfs/lib/*:
    /opt/mapr/hadoop/hadoop-2.5.1/share/hadoop/hdfs/*:/opt/mapr/
    hadoop/hadoop-2.5.1/share/hadoop/yarn/lib/*:/opt/mapr/
    hadoop/hadoop-2.5.1/share/hadoop/yarn/*:/opt/mapr/hadoop/hadoop-2.5.1/share/
    hadoop/mapreduce/lib/*:/opt/mapr/hadoop/hadoop-2.5.1/
    share/hadoop/mapreduce/*

    Thanks,
    Jilani
    On Sat, Jun 11, 2016 at 6:39 AM, Ted Yu wrote:

    Which version of hbase / Hadoop are you using ?
    (So that line number matching can be more accurate)

    It would be good if you can show your code snippet.

    Thanks

    On Jun 11, 2016, at 12:57 AM, Jilani Shaik wrote:

    Hi,

    I am trying to do hbase table bulk load from data file using map reduce.
    in the process I am getting the Null pointer exception during the shuffle
    and sort. I have both mapper and reducer in the flow. where as mapper is
    ending with success and reducer not being invoked even setup.

    It is showing the below error in terminal where the job executed.

    Error: java.lang.NullPointerException

    at
    com.google.common.base.Preconditions.checkNotNull(Preconditions.java:191)

    at com.google.common.base.Splitter.split(Splitter.java:375)

    at
    org.apache.hadoop.hbase.mapreduce.ImportTsv$TsvParser.<init>(ImportTsv.java:151)

    at
    org.apache.hadoop.hbase.mapreduce.TextSortReducer.setup(TextSortReducer.java:105)

    at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:168)

    at
    org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:627)

    at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:389)

    at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)

    at java.security.AccessController.doPrivileged(Native Method)

    at javax.security.auth.Subject.doAs(Subject.java:415)

    at
    org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1566)

    at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)



    Please let me know what could be the issue.


    Thanks,
    Jilani

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupuser @
categorieshbase, hadoop
postedJun 11, '16 at 11:40a
activeJun 15, '16 at 6:05p
posts2
users2
websitehbase.apache.org

2 users in discussion

Jilani Shaik: 1 post Ted Yu: 1 post

People

Translate

site design / logo © 2018 Grokbase