FAQ
Hi All,

I have installed Hadoop using Cloudera Manager on 8 VMs in Google Computing
Engine.

I have several doubts regarding this installation.
Is it true that the manager will install Hadoop on all 8 VMs?
I wanted to test the installation so while I was searching for some
examples, I found the quick start guide in
https://ccp.cloudera.com/display/CDH4DOC/CDH4+Documentation. I wanted to
use the example given in page 13. However, I'm not able to run from that
step. I had to install "hadoop-0.20-conf-pseudo". I'm not sure as to why I
have to install Hadoop again.

When I followed, I could start Hadoop but again stuck in starting
MapReduce. It throws an error saying "Error: JAVA_HOME is not set and Java
could not be found". However, java is installed and echo $JAVA_HOME is
giving a path.

Please tell me as to what is happening and how do I test Hadoop
installation. I have only one more week of GCE access and need to run an
application before that.

Thanks,
Ranjani

Search Discussions

  • Joey Echeverria at Oct 22, 2012 at 4:07 pm
    Hi Ranjani,

    Since you're installing with Cloudera Manager, you should not install
    the hadoop-0.20-conf-pseudo package. That's likely to leave you with
    the wrong configuration.

    What you want to do instead is to go the CM GUI and tell it to deploy
    the client configuration
    (https://ccp.cloudera.com/display/FREE4DOC/Deploying+Client+Configuration+Files#DeployingClientConfigurationFiles-DeployingtheClientConfigurationFiles).

    Please do that and try again.

    If you're still having issues, please send more details on how you
    installed and what your current configuration looks like (number of
    nodes, IP scheme, DNS setup, etc.).

    -Joey
    On Mon, Oct 22, 2012 at 4:41 AM, Ranjani wrote:
    Hi All,

    I have installed Hadoop using Cloudera Manager on 8 VMs in Google Computing
    Engine.

    I have several doubts regarding this installation.
    Is it true that the manager will install Hadoop on all 8 VMs?
    I wanted to test the installation so while I was searching for some
    examples, I found the quick start guide in
    https://ccp.cloudera.com/display/CDH4DOC/CDH4+Documentation. I wanted to use
    the example given in page 13. However, I'm not able to run from that step. I
    had to install "hadoop-0.20-conf-pseudo". I'm not sure as to why I have to
    install Hadoop again.

    When I followed, I could start Hadoop but again stuck in starting MapReduce.
    It throws an error saying "Error: JAVA_HOME is not set and Java could not be
    found". However, java is installed and echo $JAVA_HOME is giving a path.

    Please tell me as to what is happening and how do I test Hadoop
    installation. I have only one more week of GCE access and need to run an
    application before that.

    Thanks,
    Ranjani


    --
    Joey Echeverria
    Principal Solutions Architect
    Cloudera, Inc.
  • Ranjani at Oct 23, 2012 at 2:20 am
    Hi,

    Ok I will try that. But by mistake I installed hadoop-0.20-conf-pseudo
    package. Can I just uninstall the package? Will that alone suffice or do I
    have to change some settings so that it works correctly through Cloudera
    Manager.

    Thanks,
    Ranjani
    On Tuesday, October 23, 2012 12:07:31 AM UTC+8, Joey Echeverria wrote:

    Hi Ranjani,

    Since you're installing with Cloudera Manager, you should not install
    the hadoop-0.20-conf-pseudo package. That's likely to leave you with
    the wrong configuration.

    What you want to do instead is to go the CM GUI and tell it to deploy
    the client configuration
    (
    https://ccp.cloudera.com/display/FREE4DOC/Deploying+Client+Configuration+Files#DeployingClientConfigurationFiles-DeployingtheClientConfigurationFiles).


    Please do that and try again.

    If you're still having issues, please send more details on how you
    installed and what your current configuration looks like (number of
    nodes, IP scheme, DNS setup, etc.).

    -Joey
    On Mon, Oct 22, 2012 at 4:41 AM, Ranjani wrote:
    Hi All,

    I have installed Hadoop using Cloudera Manager on 8 VMs in Google Computing
    Engine.

    I have several doubts regarding this installation.
    Is it true that the manager will install Hadoop on all 8 VMs?
    I wanted to test the installation so while I was searching for some
    examples, I found the quick start guide in
    https://ccp.cloudera.com/display/CDH4DOC/CDH4+Documentation. I wanted to use
    the example given in page 13. However, I'm not able to run from that step. I
    had to install "hadoop-0.20-conf-pseudo". I'm not sure as to why I have to
    install Hadoop again.

    When I followed, I could start Hadoop but again stuck in starting
    MapReduce.
    It throws an error saying "Error: JAVA_HOME is not set and Java could not be
    found". However, java is installed and echo $JAVA_HOME is giving a path.

    Please tell me as to what is happening and how do I test Hadoop
    installation. I have only one more week of GCE access and need to run an
    application before that.

    Thanks,
    Ranjani


    --
    Joey Echeverria
    Principal Solutions Architect
    Cloudera, Inc.
  • Joey Echeverria at Oct 23, 2012 at 2:45 am
    The /etc/hadoop/conf directory is managed by alternatives, so simply
    deploying the client configurations will fix it. It's safe to either
    leave the package in place or remove it, your choice.

    -Joey
    On Mon, Oct 22, 2012 at 10:19 PM, Ranjani wrote:

    Hi,

    Ok I will try that. But by mistake I installed hadoop-0.20-conf-pseudo
    package. Can I just uninstall the package? Will that alone suffice or do I
    have to change some settings so that it works correctly through Cloudera
    Manager.

    Thanks,
    Ranjani
    On Tuesday, October 23, 2012 12:07:31 AM UTC+8, Joey Echeverria wrote:

    Hi Ranjani,

    Since you're installing with Cloudera Manager, you should not install
    the hadoop-0.20-conf-pseudo package. That's likely to leave you with
    the wrong configuration.

    What you want to do instead is to go the CM GUI and tell it to deploy
    the client configuration

    (https://ccp.cloudera.com/display/FREE4DOC/Deploying+Client+Configuration+Files#DeployingClientConfigurationFiles-DeployingtheClientConfigurationFiles).

    Please do that and try again.

    If you're still having issues, please send more details on how you
    installed and what your current configuration looks like (number of
    nodes, IP scheme, DNS setup, etc.).

    -Joey
    On Mon, Oct 22, 2012 at 4:41 AM, Ranjani wrote:
    Hi All,

    I have installed Hadoop using Cloudera Manager on 8 VMs in Google
    Computing
    Engine.

    I have several doubts regarding this installation.
    Is it true that the manager will install Hadoop on all 8 VMs?
    I wanted to test the installation so while I was searching for some
    examples, I found the quick start guide in
    https://ccp.cloudera.com/display/CDH4DOC/CDH4+Documentation. I wanted to
    use
    the example given in page 13. However, I'm not able to run from that
    step. I
    had to install "hadoop-0.20-conf-pseudo". I'm not sure as to why I have
    to
    install Hadoop again.

    When I followed, I could start Hadoop but again stuck in starting
    MapReduce.
    It throws an error saying "Error: JAVA_HOME is not set and Java could
    not be
    found". However, java is installed and echo $JAVA_HOME is giving a path.

    Please tell me as to what is happening and how do I test Hadoop
    installation. I have only one more week of GCE access and need to run an
    application before that.

    Thanks,
    Ranjani


    --
    Joey Echeverria
    Principal Solutions Architect
    Cloudera, Inc.


    --
    Joey Echeverria
    Principal Solutions Architect
    Cloudera, Inc.
  • Ranjani at Oct 29, 2012 at 1:39 am
    Hi,

    I'm not getting the JAVA_HOME error now. I was able to run the example
    given in
    https://ccp.cloudera.com/display/CDH2DOC/Running+MapReduce+Jobs without
    errors.
    However, when I run the command hadoop jar
    $HADOOP_HOME/hadoop-*-examples.jar wordcount input output1
    I get the following output

    12/10/24 02:44:43 WARN mapred.JobClient: Use GenericOptionsParser for
    parsing the arguments. Applications should implement Tool for the same.
    12/10/24 02:44:43 INFO input.FileInputFormat: Total input paths to process
    : 2
    12/10/24 02:44:43 WARN snappy.LoadSnappy: Snappy native library is available
    12/10/24 02:44:43 INFO snappy.LoadSnappy: Snappy native library loaded
    12/10/24 02:44:44 INFO mapred.JobClient: Running job: job_201210160728_0005
    12/10/24 02:44:45 INFO mapred.JobClient: map 0% reduce 0%
    12/10/24 02:44:45 INFO mapred.JobClient: Job complete: job_201210160728_0005
    12/10/24 02:44:45 INFO mapred.JobClient: Counters: 0

    I'm running the Hadoop on a Google compute engine with 1 SecondaryNameNode<http://108.59.82.114:7180/cmf/services/8/instances/36/status>,
    1 NameNode <http://108.59.82.114:7180/cmf/services/8/instances/34/status>, 1
    Balancer <http://108.59.82.114:7180/cmf/services/8/instances/35/status>, 8
    DataNodes<http://108.59.82.114:7180/cmf/services/8/instances?filterRoleType=DATANODE> .


    I also want to know where the input and output files are located and how
    can I check for logs.

    Thanks,
    Ranjani


    On Tuesday, October 23, 2012 10:45:20 AM UTC+8, Joey Echeverria wrote:

    The /etc/hadoop/conf directory is managed by alternatives, so simply
    deploying the client configurations will fix it. It's safe to either
    leave the package in place or remove it, your choice.

    -Joey
    On Mon, Oct 22, 2012 at 10:19 PM, Ranjani wrote:

    Hi,

    Ok I will try that. But by mistake I installed hadoop-0.20-conf-pseudo
    package. Can I just uninstall the package? Will that alone suffice or do I
    have to change some settings so that it works correctly through Cloudera
    Manager.

    Thanks,
    Ranjani
    On Tuesday, October 23, 2012 12:07:31 AM UTC+8, Joey Echeverria wrote:

    Hi Ranjani,

    Since you're installing with Cloudera Manager, you should not install
    the hadoop-0.20-conf-pseudo package. That's likely to leave you with
    the wrong configuration.

    What you want to do instead is to go the CM GUI and tell it to deploy
    the client configuration

    (
    https://ccp.cloudera.com/display/FREE4DOC/Deploying+Client+Configuration+Files#DeployingClientConfigurationFiles-DeployingtheClientConfigurationFiles).
    Please do that and try again.

    If you're still having issues, please send more details on how you
    installed and what your current configuration looks like (number of
    nodes, IP scheme, DNS setup, etc.).

    -Joey
    On Mon, Oct 22, 2012 at 4:41 AM, Ranjani wrote:
    Hi All,

    I have installed Hadoop using Cloudera Manager on 8 VMs in Google
    Computing
    Engine.

    I have several doubts regarding this installation.
    Is it true that the manager will install Hadoop on all 8 VMs?
    I wanted to test the installation so while I was searching for some
    examples, I found the quick start guide in
    https://ccp.cloudera.com/display/CDH4DOC/CDH4+Documentation. I
    wanted to
    use
    the example given in page 13. However, I'm not able to run from that
    step. I
    had to install "hadoop-0.20-conf-pseudo". I'm not sure as to why I
    have
    to
    install Hadoop again.

    When I followed, I could start Hadoop but again stuck in starting
    MapReduce.
    It throws an error saying "Error: JAVA_HOME is not set and Java could
    not be
    found". However, java is installed and echo $JAVA_HOME is giving a
    path.
    Please tell me as to what is happening and how do I test Hadoop
    installation. I have only one more week of GCE access and need to run
    an
    application before that.

    Thanks,
    Ranjani


    --
    Joey Echeverria
    Principal Solutions Architect
    Cloudera, Inc.


    --
    Joey Echeverria
    Principal Solutions Architect
    Cloudera, Inc.
  • Lexman at Mar 28, 2013 at 11:30 am
    I know this is an old thread, but I am posting this as I am getting the
    same error message while starting MapReduce. (STEP 6 of below link) I have
    used the below URL from cloudera..I am trying to test with single linux
    node..(ubuntu 12.04)

    https://ccp.cloudera.com/display/CDH4DOC/Installing+CDH4+on+a+Single+Linux+Node+in+Pseudo-distributed+Mode#InstallingCDH4onaSingleLinuxNodeinPseudo-distributedMode-InstallingCDH4withMRv1onaSingleLinuxNodeinPseudodistributedmode

    Additional info:
    a) echo JAVA_HOME is displaying correctly and using version jdk1.6.0_31
    b) HDFS is showing correctly in localhost


    On Monday, October 22, 2012 9:45:20 PM UTC-5, Joey Echeverria wrote:

    The /etc/hadoop/conf directory is managed by alternatives, so simply
    deploying the client configurations will fix it. It's safe to either
    leave the package in place or remove it, your choice.

    -Joey
    On Mon, Oct 22, 2012 at 10:19 PM, Ranjani wrote:

    Hi,

    Ok I will try that. But by mistake I installed hadoop-0.20-conf-pseudo
    package. Can I just uninstall the package? Will that alone suffice or do I
    have to change some settings so that it works correctly through Cloudera
    Manager.

    Thanks,
    Ranjani
    On Tuesday, October 23, 2012 12:07:31 AM UTC+8, Joey Echeverria wrote:

    Hi Ranjani,

    Since you're installing with Cloudera Manager, you should not install
    the hadoop-0.20-conf-pseudo package. That's likely to leave you with
    the wrong configuration.

    What you want to do instead is to go the CM GUI and tell it to deploy
    the client configuration

    (
    https://ccp.cloudera.com/display/FREE4DOC/Deploying+Client+Configuration+Files#DeployingClientConfigurationFiles-DeployingtheClientConfigurationFiles).
    Please do that and try again.

    If you're still having issues, please send more details on how you
    installed and what your current configuration looks like (number of
    nodes, IP scheme, DNS setup, etc.).

    -Joey
    On Mon, Oct 22, 2012 at 4:41 AM, Ranjani wrote:
    Hi All,

    I have installed Hadoop using Cloudera Manager on 8 VMs in Google
    Computing
    Engine.

    I have several doubts regarding this installation.
    Is it true that the manager will install Hadoop on all 8 VMs?
    I wanted to test the installation so while I was searching for some
    examples, I found the quick start guide in
    https://ccp.cloudera.com/display/CDH4DOC/CDH4+Documentation. I
    wanted to
    use
    the example given in page 13. However, I'm not able to run from that
    step. I
    had to install "hadoop-0.20-conf-pseudo". I'm not sure as to why I
    have
    to
    install Hadoop again.

    When I followed, I could start Hadoop but again stuck in starting
    MapReduce.
    It throws an error saying "Error: JAVA_HOME is not set and Java could
    not be
    found". However, java is installed and echo $JAVA_HOME is giving a
    path.
    Please tell me as to what is happening and how do I test Hadoop
    installation. I have only one more week of GCE access and need to run
    an
    application before that.

    Thanks,
    Ranjani


    --
    Joey Echeverria
    Principal Solutions Architect
    Cloudera, Inc.


    --
    Joey Echeverria
    Principal Solutions Architect
    Cloudera, Inc.
  • Ranjani at Oct 29, 2012 at 1:39 am
    Hi,

    I'm not getting the JAVA_HOME error now. I was able to run the example
    given in
    https://ccp.cloudera.com/display/CDH2DOC/Running+MapReduce+Jobs without
    errors.
    However, when I run the command hadoop jar
    $HADOOP_HOME/hadoop-*-examples.jar wordcount input output1
    I get the following output

    12/10/24 02:44:43 WARN mapred.JobClient: Use GenericOptionsParser for
    parsing the arguments. Applications should implement Tool for the same.
    12/10/24 02:44:43 INFO input.FileInputFormat: Total input paths to process
    : 2
    12/10/24 02:44:43 WARN snappy.LoadSnappy: Snappy native library is available
    12/10/24 02:44:43 INFO snappy.LoadSnappy: Snappy native library loaded
    12/10/24 02:44:44 INFO mapred.JobClient: Running job: job_201210160728_0005
    12/10/24 02:44:45 INFO mapred.JobClient: map 0% reduce 0%
    12/10/24 02:44:45 INFO mapred.JobClient: Job complete: job_201210160728_0005
    12/10/24 02:44:45 INFO mapred.JobClient: Counters: 0

    I'm running the Hadoop on a Google compute engine with 1 SecondaryNameNode<http://108.59.82.114:7180/cmf/services/8/instances/36/status>,
    1 NameNode <http://108.59.82.114:7180/cmf/services/8/instances/34/status>, 1
    Balancer <http://108.59.82.114:7180/cmf/services/8/instances/35/status>, 8
    DataNodes<http://108.59.82.114:7180/cmf/services/8/instances?filterRoleType=DATANODE> .


    I also want to know where the input and output files are located and how
    can I check for logs.

    Thanks,
    Ranjani
    On Monday, October 22, 2012 4:41:45 PM UTC+8, Ranjani wrote:

    Hi All,

    I have installed Hadoop using Cloudera Manager on 8 VMs in Google
    Computing Engine.

    I have several doubts regarding this installation.
    Is it true that the manager will install Hadoop on all 8 VMs?
    I wanted to test the installation so while I was searching for some
    examples, I found the quick start guide in
    https://ccp.cloudera.com/display/CDH4DOC/CDH4+Documentation. I wanted to
    use the example given in page 13. However, I'm not able to run from that
    step. I had to install "hadoop-0.20-conf-pseudo". I'm not sure as to why I
    have to install Hadoop again.

    When I followed, I could start Hadoop but again stuck in starting
    MapReduce. It throws an error saying "Error: JAVA_HOME is not set and Java
    could not be found". However, java is installed and echo $JAVA_HOME is
    giving a path.

    Please tell me as to what is happening and how do I test Hadoop
    installation. I have only one more week of GCE access and need to run an
    application before that.

    Thanks,
    Ranjani

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupscm-users @
categorieshadoop
postedOct 22, '12 at 8:47a
activeMar 28, '13 at 11:30a
posts7
users3
websitecloudera.com
irc#hadoop

People

Translate

site design / logo © 2022 Grokbase