Hello Clodera Manager Users,
I am new to Cloudera Manager or even hadoop. I installed the CDH 4.4 using
Cloudera manager using path A installation method and facing some BAD
HEALTH issues mainly due to connectivity isses between nodes. I have tried
this thrice but failed to solve this problem. I request the experts in the
forum to help me get through this. Your help is really appreciated.
I have Centos 6 64 bit running on 4 VMs.
Cluster configuration that i wanted is 1 Master node (Name & Sec Name)
which serves as data node and another 3 data nodes on different VMs.
Before installing Cloudera Manager, below are the network setting that i
made.
*Step 1:*
all my hosts has user with same name "hadoop" and for the sake of password
less sudo login for Cloudera manager, i added ALL ALL = (ALL) NOPASSWD:
ALL in my /etc/sudoers in all my nodes for time being. Idea is to revert
this after CDH cluster installation.
*Step 2:*
In /etc/sysconfig/network i added the HOSTNAME=master.com / slave1.com /
slave2.com / slave3.com in corresponding VM instances.
*Step 3:*
In /etc/hosts i added host names of every node like..
192.168.1.1 master.com master
192.168.1.2 slave1.com slave1
192.168.1.3 slave2.com slave2
192.168.1.4 slave3.com slave3
*Step 4:*
Generated ssh key ( ssh-keygen -t rsa -P '' -f ~/.ssh/id_rsa ) and made
public key of each node available to all nodes in authorized_keys file.
Note: after above step, i am able to do ssh from/to any node with out
password on a terminal.
Step 5:
Installed CDH 4.4 using cloudera manager successfully, using path A
installation method with default embedded postgress db.
But after installation, the home page of my cloudera manager web console, i
could see my hdfs, mapreduce1, hbase & other few serves showing bad health.
When i clicked to see details, i could see the reason is because if
connectivity between nodes. Data nodes not able to connect with name nodes.
Attaching the screen shot for your reference.
<https://lh3.googleusercontent.com/-zo3kFNJGBO0/UjTJ-DRSaUI/AAAAAAAABb8/7u03mdNp-LE/s1600/CDH+cluster_BadHealth.jpg>
This is the server log..
Caused by: java.net.NoRouteToHostException: No route to host
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:207)
at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:528)
at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:492)
at org.apache.hadoop.ipc.Client$Connection.setupConnection(Client.java:509)
at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:603)
at org.apache.hadoop.ipc.Client$Connection.access$2100(Client.java:252)
at org.apache.hadoop.ipc.Client.getConnection(Client.java:1290)
at org.apache.hadoop.ipc.Client.call(Client.java:1208)
and also
Caused by: java.lang.NullPointerException
at org.apache.hadoop.hdfs.server.datanode.DataNode.getVolumeInfo(DataNode.java:2344)
at sun.reflect.GeneratedMethodAccessor80.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at com.sun.jmx.mbeanserver.ConvertingMethod.invokeWithOpenReturn(ConvertingMethod.java:167)
at com.sun.jmx.mbeanserver.MXBeanIntrospector.invokeM2(MXBeanIntrospector.java:96)
at com.sun.jmx.mbeanserver.MXBeanIntrospector.invokeM2(MXBeanIntrospector.java:33)
at com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:208)
at com.sun.jmx.mbeanserver.PerInterface.getAttribute(PerInterface.java:65)
at com.sun.jmx.mbeanserver.MBeanSupport.getAttribute(MBeanSupport.java:216)
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getAttribute(DefaultMBeanServerInterceptor.java:666)
Could anybody please help me.
Thanks in advance
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
