FAQ
Hi!

I'd like to know how to fuse mount hdfs on HA(High Availability) name node
in CDH4.

I am trying to test hdfs HA on CDH4.
I confugre CDH4 hadoop cluster which is consist of 7 server(1 Cloudera
Manager server, 2 name node, 4 data node)

Before Configuring HA Cluster, I mount hdfs with fuse. using follow command.
hadoop-fuse-dfs dfs://name01:8020 /cdh4
But I can't mount when I configure HA Cluster.

core-site.xml
----------------------------------------
<?xml version="1.0" encoding="UTF-8" ?>
*-* <!--

Autogenerated by Cloudera CM on 2012-07-05T03:48:50.462Z

* * -->
*-*<file:///D:/BigData/Cloudera/hdfs1-clientconfig_auto/hadoop-conf/core-site.xml#>
<configuration>
*-*<file:///D:/BigData/Cloudera/hdfs1-clientconfig_auto/hadoop-conf/core-site.xml#>
<property>
* * <name>*fs.defaultFS*</name>
* * <value>*hdfs://nameservice1*</value>
* * </property>
*-*<file:///D:/BigData/Cloudera/hdfs1-clientconfig_auto/hadoop-conf/core-site.xml#>
<property>
* * <name>*hadoop.security.authentication*</name>
* * <value>*simple*</value>
* * </property>
*-*<file:///D:/BigData/Cloudera/hdfs1-clientconfig_auto/hadoop-conf/core-site.xml#>
<property>
* * <name>*hadoop.security.auth_to_local*</name>
* * <value>*DEFAULT*</value>
* * </property>
* * </configuration>
--------------------------------------------------------------

hdfs-site.xml
--------------------------------------------------------------
* * <?xml version="1.0" encoding="UTF-8" ?>
*-* <!--

Autogenerated by Cloudera CM on 2012-07-05T03:48:50.455Z

* * -->
*-*<file:///D:/BigData/Cloudera/hdfs1-clientconfig_auto/hadoop-conf/hdfs-site.xml#>
<configuration>
*-*<file:///D:/BigData/Cloudera/hdfs1-clientconfig_auto/hadoop-conf/hdfs-site.xml#>
<property>
* * <name>*dfs.nameservices*</name>
* * <value>*nameservice1*</value>
* * </property>
*-*<file:///D:/BigData/Cloudera/hdfs1-clientconfig_auto/hadoop-conf/hdfs-site.xml#>
<property>
* * <name>*dfs.client.failover.proxy.provider.nameservice1*</name>
* * <value>*
org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider*</
value>
* * </property>
*-*<file:///D:/BigData/Cloudera/hdfs1-clientconfig_auto/hadoop-conf/hdfs-site.xml#>
<property>
* * <name>*dfs.ha.automatic-failover.enabled.nameservice1*</name>
* * <value>*true*</value>
* * </property>
*-*<file:///D:/BigData/Cloudera/hdfs1-clientconfig_auto/hadoop-conf/hdfs-site.xml#>
<property>
* * <name>*ha.zookeeper.quorum*</name>
* * <value>*name02:2181,name01:2181,data00:2181*</value>
* * </property>
*-*<file:///D:/BigData/Cloudera/hdfs1-clientconfig_auto/hadoop-conf/hdfs-site.xml#>
<property>
* * <name>*dfs.ha.namenodes.nameservice1*</name>
* * <value>*namenode73,namenode49*</value>
* * </property>
*-*<file:///D:/BigData/Cloudera/hdfs1-clientconfig_auto/hadoop-conf/hdfs-site.xml#>
<property>
* * <name>*dfs.namenode.rpc-address.nameservice1.namenode73*</name>
* * <value>*name02:8020*</value>
* * </property>
*-*<file:///D:/BigData/Cloudera/hdfs1-clientconfig_auto/hadoop-conf/hdfs-site.xml#>
<property>
* * <name>*dfs.namenode.http-address.nameservice1.namenode73*</name>
* * <value>*name02:50070*</value>
* * </property>
*-*<file:///D:/BigData/Cloudera/hdfs1-clientconfig_auto/hadoop-conf/hdfs-site.xml#>
<property>
* * <name>*dfs.namenode.https-address.nameservice1.namenode73*</name>
* * <value>*name02:50470*</value>
* * </property>
*-*<file:///D:/BigData/Cloudera/hdfs1-clientconfig_auto/hadoop-conf/hdfs-site.xml#>
<property>
* * <name>*dfs.namenode.rpc-address.nameservice1.namenode49*</name>
* * <value>*name01:8020*</value>
* * </property>
*-*<file:///D:/BigData/Cloudera/hdfs1-clientconfig_auto/hadoop-conf/hdfs-site.xml#>
<property>
* * <name>*dfs.namenode.http-address.nameservice1.namenode49*</name>
* * <value>*name01:50070*</value>
* * </property>
*-*<file:///D:/BigData/Cloudera/hdfs1-clientconfig_auto/hadoop-conf/hdfs-site.xml#>
<property>
* * <name>*dfs.namenode.https-address.nameservice1.namenode49*</name>
* * <value>*name01:50470*</value>
* * </property>
*-*<file:///D:/BigData/Cloudera/hdfs1-clientconfig_auto/hadoop-conf/hdfs-site.xml#>
<property>
* * <name>*dfs.replication*</name>
* * <value>*3*</value>
* * </property>
*-*<file:///D:/BigData/Cloudera/hdfs1-clientconfig_auto/hadoop-conf/hdfs-site.xml#>
<property>
* * <name>*dfs.blocksize*</name>
* * <value>*134217728*</value>
* * </property>
*-*<file:///D:/BigData/Cloudera/hdfs1-clientconfig_auto/hadoop-conf/hdfs-site.xml#>
<property>
* * <name>*dfs.client.use.datanode.hostname*</name>
* * <value>*false*</value>
* * </property>
* * </configuration>
------------------------------------------------------------------

Search Discussions

  • Harsh J at Jul 7, 2012 at 2:18 pm
    Hi,

    There's a bug around this part that will be addressed by
    https://issues.apache.org/jira/browse/HDFS-3609 and others soon. For
    the moment, if you pass FuseDFS a special config file instead, which
    has a proper URI format it expects due to the bug (hdfs://host:port),
    of a single NN, it should let you mount it.
    On Thu, Jul 5, 2012 at 11:40 AM, jason wrote:
    Hi!

    I'd like to know how to fuse mount hdfs on HA(High Availability) name node
    in CDH4.

    I am trying to test hdfs HA on CDH4.
    I confugre CDH4 hadoop cluster which is consist of 7 server(1 Cloudera
    Manager server, 2 name node, 4 data node)

    Before Configuring HA Cluster, I mount hdfs with fuse. using follow command.
    hadoop-fuse-dfs dfs://name01:8020 /cdh4
    But I can't mount when I configure HA Cluster.

    core-site.xml
    ----------------------------------------
    <?xml version="1.0" encoding="UTF-8" ?>
    - <!--

    Autogenerated by Cloudera CM on 2012-07-05T03:48:50.462Z

    -->
    - <configuration>
    - <property>
    <name>fs.defaultFS</name>
    <value>hdfs://nameservice1</value>
    </property>
    - <property>
    <name>hadoop.security.authentication</name>
    <value>simple</value>
    </property>
    - <property>
    <name>hadoop.security.auth_to_local</name>
    <value>DEFAULT</value>
    </property>
    </configuration>
    --------------------------------------------------------------

    hdfs-site.xml
    --------------------------------------------------------------
    <?xml version="1.0" encoding="UTF-8" ?>
    - <!--

    Autogenerated by Cloudera CM on 2012-07-05T03:48:50.455Z

    -->
    - <configuration>
    - <property>
    <name>dfs.nameservices</name>
    <value>nameservice1</value>
    </property>
    - <property>
    <name>dfs.client.failover.proxy.provider.nameservice1</name>

    <value>org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider</value>
    </property>
    - <property>
    <name>dfs.ha.automatic-failover.enabled.nameservice1</name>
    <value>true</value>
    </property>
    - <property>
    <name>ha.zookeeper.quorum</name>
    <value>name02:2181,name01:2181,data00:2181</value>
    </property>
    - <property>
    <name>dfs.ha.namenodes.nameservice1</name>
    <value>namenode73,namenode49</value>
    </property>
    - <property>
    <name>dfs.namenode.rpc-address.nameservice1.namenode73</name>
    <value>name02:8020</value>
    </property>
    - <property>
    <name>dfs.namenode.http-address.nameservice1.namenode73</name>
    <value>name02:50070</value>
    </property>
    - <property>
    <name>dfs.namenode.https-address.nameservice1.namenode73</name>
    <value>name02:50470</value>
    </property>
    - <property>
    <name>dfs.namenode.rpc-address.nameservice1.namenode49</name>
    <value>name01:8020</value>
    </property>
    - <property>
    <name>dfs.namenode.http-address.nameservice1.namenode49</name>
    <value>name01:50070</value>
    </property>
    - <property>
    <name>dfs.namenode.https-address.nameservice1.namenode49</name>
    <value>name01:50470</value>
    </property>
    - <property>
    <name>dfs.replication</name>
    <value>3</value>
    </property>
    - <property>
    <name>dfs.blocksize</name>
    <value>134217728</value>
    </property>
    - <property>
    <name>dfs.client.use.datanode.hostname</name>
    <value>false</value>
    </property>
    </configuration>
    ------------------------------------------------------------------





    --
    Harsh J

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcdh-user @
categorieshadoop
postedJul 6, '12 at 12:50a
activeJul 7, '12 at 2:18p
posts2
users2
websitecloudera.com
irc#hadoop

2 users in discussion

Jason: 1 post Harsh J: 1 post

People

Translate

site design / logo © 2022 Grokbase