FAQ
I'm trying to install Cloudera Manager 4.1 in an official Cloudera 4.1 VM I
got from Cloudera's website, using CentOS.

I downloaded the binary installer, it fails by putting the following into a
log file:

runuser: user cloudera-scm does not exist

I'm running the installer as root. I also tried creating a cloudera-scm
user by hand, didn't help.

Search Discussions

  • Saurabh Agarwal at Nov 21, 2012 at 12:08 pm
    Hi,

    I need to add the external jar files in map reduce job. I am using the "-libjars" option -

    Hadoop jar myjob.jar jobClassName -libjars external.jar

    It throws that classes inside external.jar was not found. I checked the tasktracker distcache folder at hadoop node and did not find external jar file there.

    Does external.jar has to be in classpath of Hadoop program?

    What am I missing here?

    Thanks in advance for your help.

    Saurabh.

    >
  • Harsh J at Nov 21, 2012 at 1:05 pm
    Hey Saurabh,

    When using CLI options such as -libjars (via the Tool, Configured and
    ToolRunner utilities), ensure you reuse the provided Configuration
    object (via a getConf() call) and not a new Configuration object. Can
    you check for this?
    On Wed, Nov 21, 2012 at 5:30 PM, Saurabh Agarwal wrote:
    Hi,

    I need to add the external jar files in map reduce job. I am using the "-libjars" option -

    Hadoop jar myjob.jar jobClassName -libjars external.jar

    It throws that classes inside external.jar was not found. I checked the tasktracker distcache folder at hadoop node and did not find external jar file there.

    Does external.jar has to be in classpath of Hadoop program?

    What am I missing here?

    Thanks in advance for your help.

    Saurabh.


    --
    Harsh J
  • Saurabh Agarwal at Nov 21, 2012 at 1:18 pm
    Thanks. In my main program, I am passing new configuration to ToolRunner program. Let me make that change.
    On Nov 21, 2012, at 8:05 AM, Harsh J wrote:

    Hey Saurabh,

    When using CLI options such as -libjars (via the Tool, Configured and
    ToolRunner utilities), ensure you reuse the provided Configuration
    object (via a getConf() call) and not a new Configuration object. Can
    you check for this?
    On Wed, Nov 21, 2012 at 5:30 PM, Saurabh Agarwal wrote:
    Hi,

    I need to add the external jar files in map reduce job. I am using the "-libjars" option -

    Hadoop jar myjob.jar jobClassName -libjars external.jar

    It throws that classes inside external.jar was not found. I checked the tasktracker distcache folder at hadoop node and did not find external jar file there.

    Does external.jar has to be in classpath of Hadoop program?

    What am I missing here?

    Thanks in advance for your help.

    Saurabh.


    --
    Harsh J
  • Saurabh Agarwal at Nov 21, 2012 at 6:40 pm
    Hi Harsh,

    I checked the code. I am using the getconf() inside the run method. I am still getting the exception. Do I need to make any changes.

    Let me know,
    Saurabh.

    Here is the code snippet.

    public static void main(String[] args)throws Exception {

    int res = ToolRunner.run(newConfiguration(), new MapReduce(), args);
    System.exit(res);
    }

    @Override
    public int run(String[] args) throwsException {

    // Configuration processed by ToolRunner
    Configuration conf = getConf();

    Job job = new Job (conf, "MapReduce");
    On Nov 21, 2012, at 8:05 AM, Harsh J wrote:

    Hey Saurabh,

    When using CLI options such as -libjars (via the Tool, Configured and
    ToolRunner utilities), ensure you reuse the provided Configuration
    object (via a getConf() call) and not a new Configuration object. Can
    you check for this?
    On Wed, Nov 21, 2012 at 5:30 PM, Saurabh Agarwal wrote:
    Hi,

    I need to add the external jar files in map reduce job. I am using the "-libjars" option -

    Hadoop jar myjob.jar jobClassName -libjars external.jar

    It throws that classes inside external.jar was not found. I checked the tasktracker distcache folder at hadoop node and did not find external jar file there.

    Does external.jar has to be in classpath of Hadoop program?

    What am I missing here?

    Thanks in advance for your help.

    Saurabh.


    --
    Harsh J
  • Joey Echeverria at Nov 21, 2012 at 9:26 pm
    Where does it throw? -libjars only adds the classes for the remote
    tasks, if you need to use a class in the run() method, you have to
    also add it to HADOOP_CLASSPATH.

    -Joey
    On Wed, Nov 21, 2012 at 1:40 PM, Saurabh Agarwal wrote:
    Hi Harsh,


    I checked the code. I am using the getconf() inside the run method. I am
    still getting the exception. Do I need to make any changes.


    Let me know,

    Saurabh.


    Here is the code snippet.


    public static void main(String[] args)throws Exception {



    int res = ToolRunner.run(newConfiguration(), new MapReduce(),
    args);

    System.exit(res);

    }



    @Override

    public int run(String[] args) throwsException {



    // Configuration processed by ToolRunner

    Configuration conf = getConf();



    Job job = new Job (conf, "MapReduce");


    On Nov 21, 2012, at 8:05 AM, Harsh J wrote:

    Hey Saurabh,

    When using CLI options such as -libjars (via the Tool, Configured and
    ToolRunner utilities), ensure you reuse the provided Configuration
    object (via a getConf() call) and not a new Configuration object. Can
    you check for this?

    On Wed, Nov 21, 2012 at 5:30 PM, Saurabh Agarwal wrote:

    Hi,


    I need to add the external jar files in map reduce job. I am using the
    "-libjars" option -


    Hadoop jar myjob.jar jobClassName -libjars external.jar


    It throws that classes inside external.jar was not found. I checked the
    tasktracker distcache folder at hadoop node and did not find external jar
    file there.


    Does external.jar has to be in classpath of Hadoop program?


    What am I missing here?


    Thanks in advance for your help.


    Saurabh.






    --
    Harsh J


    --
    Joey Echeverria
    Principal Solutions Architect
    Cloudera, Inc.
  • Saurabh Agarwal at Nov 21, 2012 at 9:44 pm
    Thanks. I am only adding the external library needed for remote mapper and reducer tasks.
    Hadoop jar myjob.jar jobClassName -libjars external.jar
    External jar has classes which are used by mapper and reducer.

    The above command throws -

    12/11/21 16:26:02 INFO mapred.JobClient: Task Id : attempt_201211211620_0001_m_000000_1, Status : FAILED
    Error: java.lang.ClassNotFoundException: org.joda.time.format.DateTimeFormatterBuilder.

    What am I doing wrong?

    Saurabh
    On Nov 21, 2012, at 4:26 PM, Joey Echeverria wrote:

    Where does it throw? -libjars only adds the classes for the remote
    tasks, if you need to use a class in the run() method, you have to
    also add it to HADOOP_CLASSPATH.

    -Joey
    On Wed, Nov 21, 2012 at 1:40 PM, Saurabh Agarwal wrote:
    Hi Harsh,


    I checked the code. I am using the getconf() inside the run method. I am
    still getting the exception. Do I need to make any changes.


    Let me know,

    Saurabh.


    Here is the code snippet.


    public static void main(String[] args)throws Exception {



    int res = ToolRunner.run(newConfiguration(), new MapReduce(),
    args);

    System.exit(res);

    }



    @Override

    public int run(String[] args) throwsException {



    // Configuration processed by ToolRunner

    Configuration conf = getConf();



    Job job = new Job (conf, "MapReduce");


    On Nov 21, 2012, at 8:05 AM, Harsh J wrote:

    Hey Saurabh,

    When using CLI options such as -libjars (via the Tool, Configured and
    ToolRunner utilities), ensure you reuse the provided Configuration
    object (via a getConf() call) and not a new Configuration object. Can
    you check for this?

    On Wed, Nov 21, 2012 at 5:30 PM, Saurabh Agarwal <saurabh.wl@gmail.com>
    wrote:

    Hi,


    I need to add the external jar files in map reduce job. I am using the
    "-libjars" option -


    Hadoop jar myjob.jar jobClassName -libjars external.jar


    It throws that classes inside external.jar was not found. I checked the
    tasktracker distcache folder at hadoop node and did not find external jar
    file there.


    Does external.jar has to be in classpath of Hadoop program?


    What am I missing here?


    Thanks in advance for your help.


    Saurabh.






    --
    Harsh J


    --
    Joey Echeverria
    Principal Solutions Architect
    Cloudera, Inc.
  • Mark Grover at Nov 21, 2012 at 10:46 pm
    Hi Marton,
    Sorry for the inconvenience. Cloudera Manager doesn't presently support
    taking over a previously non-managed cluster (like the one present in the
    CDH Demo VM). Here is another relevant thread from CDH Users mailing list
    with a little more info:
    https://groups.google.com/a/cloudera.org/forum/?fromgroups=#!topic/cdh-user/gM-Gk1GZSRg

    Also, for what it's worth, we are working on a demo VM that will have
    Cloudera Manager installed and configured to manage a CDH cluster on the
    same machine. That will allow you to play around with CM functionality by
    just downloading a VM image. So, please stay tuned for that.

    Mark
    On Wednesday, November 21, 2012 2:44:26 AM UTC-8, Marton Trencseni wrote:

    I'm trying to install Cloudera Manager 4.1 in an official Cloudera 4.1 VM
    I got from Cloudera's website, using CentOS.

    I downloaded the binary installer, it fails by putting the following into
    a log file:

    runuser: user cloudera-scm does not exist

    I'm running the installer as root. I also tried creating a cloudera-scm
    user by hand, didn't help.
  • Mark Grover at Dec 4, 2012 at 9:48 pm
    Hi Marton,
    Thanks for your patience. We have now released Cloudera Manager Demo VM
    which contains CM4.1.1 and CDH4.1.2. You can download it from
    https://ccp.cloudera.com/display/SUPPORT/Cloudera+Manager+Free+Edition+Demo+VM

    Mark
    On Wednesday, November 21, 2012 2:44:26 AM UTC-8, Marton Trencseni wrote:

    I'm trying to install Cloudera Manager 4.1 in an official Cloudera 4.1 VM
    I got from Cloudera's website, using CentOS.

    I downloaded the binary installer, it fails by putting the following into
    a log file:

    runuser: user cloudera-scm does not exist

    I'm running the installer as root. I also tried creating a cloudera-scm
    user by hand, didn't help.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupscm-users @
categorieshadoop
postedNov 21, '12 at 10:44a
activeDec 4, '12 at 9:48p
posts9
users5
websitecloudera.com
irc#hadoop

People

Translate

site design / logo © 2022 Grokbase