FAQ
Hi,

I am using a CDH3U3 5 node cluster set up by Cloudera Manager Free edition.
All of the node machines are running Ubuntu 10.04. I have come across a
thrift jar conflict.

I am attempting to configure a software component my company makes with the
hadoop cluster, but I believe the issue I am running into is that out
software component contains a version of Thrift that is conflicting with
the version of Thrift embedded inside the Hue Jar
(/usr/lib/hadoop-0.20/lib/hue-plugins-1.2.0-cdh3u3.jar). I have pasted what
I believe is the relevant java error below. I believe the issue I am seeing
is related to https://issues.cloudera.org/browse/HUE-584 . As a workaround,
I have moved /usr/lib/hadoop-0.20/lib/hue-plugins-1.2.0-cdh3u3.jar to a
different location and restarted the cluster. This has resloved the error I
was seeing. But I was wondering if anyone knows of a better method to avoid
or workaround this issue ?


Thank you



Caused by: java.lang.AbstractMethodError:
org.apache.thrift.TUnion.writeValue(Lorg/apache/thrift/protocol/TProtocol;)V

Search Discussions

  • Joey Echeverria at Oct 1, 2012 at 2:20 pm
    Where do you need access to your thrift library? Is it for a MapReduce job?

    If it is for MapReduce, you can use the setting that has the client
    jars take precedence in the classpath:

    http://archive.cloudera.com/cdh/3/hadoop-0.20.2-cdh3u3/api/org/apache/hadoop/mapreduce/Job.html#setUserClassesTakesPrecedence(boolean)

    There's also an environment variable you can set to make sure that
    your classpath takes precedence during job submission as well. The API
    on Job only sets it for the remote classpath.

    -Joey

    On Thu, Sep 27, 2012 at 6:21 PM, zs-rev wrote:
    Hi,

    I am using a CDH3U3 5 node cluster set up by Cloudera Manager Free edition.
    All of the node machines are running Ubuntu 10.04. I have come across a
    thrift jar conflict.

    I am attempting to configure a software component my company makes with the
    hadoop cluster, but I believe the issue I am running into is that out
    software component contains a version of Thrift that is conflicting with the
    version of Thrift embedded inside the Hue Jar
    (/usr/lib/hadoop-0.20/lib/hue-plugins-1.2.0-cdh3u3.jar). I have pasted what
    I believe is the relevant java error below. I believe the issue I am seeing
    is related to https://issues.cloudera.org/browse/HUE-584 . As a workaround,
    I have moved /usr/lib/hadoop-0.20/lib/hue-plugins-1.2.0-cdh3u3.jar to a
    different location and restarted the cluster. This has resloved the error I
    was seeing. But I was wondering if anyone knows of a better method to avoid
    or workaround this issue ?


    Thank you



    Caused by: java.lang.AbstractMethodError:
    org.apache.thrift.TUnion.writeValue(Lorg/apache/thrift/protocol/TProtocol;)V


    --
    Joey Echeverria
    Principal Solutions Architect
    Cloudera, Inc.
  • Zach Schweinfurth at Oct 1, 2012 at 3:05 pm
    Hi Joey,

    Thanks for the reply. I need access to the thrift library for a MapReduce
    job.

    What environment variable is it that you would set to ensure that your
    classpath takes precedent during the job submission?

    Also, you say that the API on Job only sets it for the remote classpath.
    Could you explain to me what is the differnce between the job submission
    and the remote classpath? How would the two methods you mention, using the
    setUserClassesTakesPrecedence and the environment varaible approach, be
    different ?

    Thank you
    On Mon, Oct 1, 2012 at 10:20 AM, Joey Echeverria wrote:

    Where do you need access to your thrift library? Is it for a MapReduce job?

    If it is for MapReduce, you can use the setting that has the client
    jars take precedence in the classpath:


    http://archive.cloudera.com/cdh/3/hadoop-0.20.2-cdh3u3/api/org/apache/hadoop/mapreduce/Job.html#setUserClassesTakesPrecedence(boolean)

    There's also an environment variable you can set to make sure that
    your classpath takes precedence during job submission as well. The API
    on Job only sets it for the remote classpath.

    -Joey

    On Thu, Sep 27, 2012 at 6:21 PM, zs-rev wrote:
    Hi,

    I am using a CDH3U3 5 node cluster set up by Cloudera Manager Free edition.
    All of the node machines are running Ubuntu 10.04. I have come across a
    thrift jar conflict.

    I am attempting to configure a software component my company makes with the
    hadoop cluster, but I believe the issue I am running into is that out
    software component contains a version of Thrift that is conflicting with the
    version of Thrift embedded inside the Hue Jar
    (/usr/lib/hadoop-0.20/lib/hue-plugins-1.2.0-cdh3u3.jar). I have pasted what
    I believe is the relevant java error below. I believe the issue I am seeing
    is related to https://issues.cloudera.org/browse/HUE-584 . As a
    workaround,
    I have moved /usr/lib/hadoop-0.20/lib/hue-plugins-1.2.0-cdh3u3.jar to a
    different location and restarted the cluster. This has resloved the error I
    was seeing. But I was wondering if anyone knows of a better method to avoid
    or workaround this issue ?


    Thank you



    Caused by: java.lang.AbstractMethodError:
    org.apache.thrift.TUnion.writeValue(Lorg/apache/thrift/protocol/TProtocol;)V



    --
    Joey Echeverria
    Principal Solutions Architect
    Cloudera, Inc.


    --
    Zachary Schweinfurth
    Ontology Engineer
    Revelytix, Inc.
  • Joey Echeverria at Oct 1, 2012 at 3:30 pm
    Sorry, I copied it but forgot to paste it:

    HADOOP_USER_CLASSPATH_FIRST

    -Joey

    On Mon, Oct 1, 2012 at 10:57 AM, Zach Schweinfurth
    wrote:
    Hi Joey,

    Thanks for the reply. I need access to the thrift library for a MapReduce
    job.

    What environment variable is it that you would set to ensure that your
    classpath takes precedent during the job submission?

    Also, you say that the API on Job only sets it for the remote classpath.
    Could you explain to me what is the differnce between the job submission and
    the remote classpath? How would the two methods you mention, using the
    setUserClassesTakesPrecedence and the environment varaible approach, be
    different ?

    Thank you

    On Mon, Oct 1, 2012 at 10:20 AM, Joey Echeverria wrote:

    Where do you need access to your thrift library? Is it for a MapReduce
    job?

    If it is for MapReduce, you can use the setting that has the client
    jars take precedence in the classpath:


    http://archive.cloudera.com/cdh/3/hadoop-0.20.2-cdh3u3/api/org/apache/hadoop/mapreduce/Job.html#setUserClassesTakesPrecedence(boolean)

    There's also an environment variable you can set to make sure that
    your classpath takes precedence during job submission as well. The API
    on Job only sets it for the remote classpath.

    -Joey


    On Thu, Sep 27, 2012 at 6:21 PM, zs-rev <zschweinfurth@revelytix.com>
    wrote:
    Hi,

    I am using a CDH3U3 5 node cluster set up by Cloudera Manager Free
    edition.
    All of the node machines are running Ubuntu 10.04. I have come across a
    thrift jar conflict.

    I am attempting to configure a software component my company makes with
    the
    hadoop cluster, but I believe the issue I am running into is that out
    software component contains a version of Thrift that is conflicting with
    the
    version of Thrift embedded inside the Hue Jar
    (/usr/lib/hadoop-0.20/lib/hue-plugins-1.2.0-cdh3u3.jar). I have pasted
    what
    I believe is the relevant java error below. I believe the issue I am
    seeing
    is related to https://issues.cloudera.org/browse/HUE-584 . As a
    workaround,
    I have moved /usr/lib/hadoop-0.20/lib/hue-plugins-1.2.0-cdh3u3.jar to a
    different location and restarted the cluster. This has resloved the
    error I
    was seeing. But I was wondering if anyone knows of a better method to
    avoid
    or workaround this issue ?


    Thank you



    Caused by: java.lang.AbstractMethodError:

    org.apache.thrift.TUnion.writeValue(Lorg/apache/thrift/protocol/TProtocol;)V


    --
    Joey Echeverria
    Principal Solutions Architect
    Cloudera, Inc.



    --
    Zachary Schweinfurth
    Ontology Engineer
    Revelytix, Inc.


    --
    Joey Echeverria
    Principal Solutions Architect
    Cloudera, Inc.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupscm-users @
categorieshadoop
postedSep 27, '12 at 10:21p
activeOct 1, '12 at 3:30p
posts4
users2
websitecloudera.com
irc#hadoop

People

Translate

site design / logo © 2022 Grokbase