FAQ
I'm configuring a local hadoop cluster in secure mode for development/experimental purposes on Ubuntu 11.04 with the hadoop-0.20.203.0 distribution from apache mirror.

I have the basic Kerberos setup working, can start namenode in secure mode and connect to it with hadoop fs -ls

I'm not able to get the datanode start in secure mode - what do I have to do to make that happen?

The error I get:

11/08/30 18:01:57 INFO security.UserGroupInformation: Login successful for user hduser/hdev-vm@HADOOP.LOCALDOMAIN using keytab file /opt/hadoop/conf/nn.keytab
11/08/30 18:01:57 ERROR datanode.DataNode: java.lang.RuntimeException: Cannot start secure cluster without privileged resources.
at org.apache.hadoop.hdfs.server.datanode.DataNode.startDataNode(DataNode.java:293)
at org.apache.hadoop.hdfs.server.datanode.DataNode.(DataNode.java:1480)
at org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:1419)
at org.apache.hadoop.hdfs.server.datanode.DataNode.createDataNode(DataNode.java:1437)
at org.apache.hadoop.hdfs.server.datanode.DataNode.secureMain(DataNode.java:1563)
at org.apache.hadoop.hdfs.server.datanode.DataNode.main(DataNode.java:1573)

11/08/30 18:01:57 INFO datanode.DataNode: SHUTDOWN_MSG:
/************************************************************
SHUTDOWN_MSG: Shutting down DataNode at hdev-vm/127.0.1.1

I have not configured the system to use port numbers that require root (yet). All I want is the datanode to run in secure mode with kerberos authentication.

Any pointers would be greatly appreciated!

Thomas

Search Discussions

  • Ravi Prakash at Aug 31, 2011 at 4:01 am
    In short you MUST use priviledged resourced.

    In long:

    Here's what I did to setup a secure single node cluster. I'm sure there's
    other ways, but here's how I did it.

    1. Install krb5-server
    2. Setup the kerberos configuration (files attached).
    /var/kerberos/krb5kdc/kdc.conf and /etc/krb5.conf
    http://yahoo.github.com/hadoop-common/installing.html
    3. To clean up everything :
    http://mailman.mit.edu/pipermail/kerberos/2003-June/003312.html
    4. Create Kerberos database $ sudo kdb5_util create -s
    5. Start Kerberos $ sudo /etc/rc.d/init.d/kadmin start $ sudo
    /etc/rc.d/init.d/krb5kdc start
    6. Create principal raviprak/localhost.localdomain@localdomain
    http://web.mit.edu/kerberos/krb5-1.5/krb5-1.5.4/doc/krb5-admin/Adding-or-Modifying-Principals.html
    7. Create keytab fiie using “xst -k /home/raviprak/raviprak.keytab
    raviprak/localhost.localdomain@localdomain”
    8. Setup hdfs-site.xml and core-site.xml (files attached)
    9. sudo hostname localhost.localdomain
    10. hadoop-daemon.sh start namenode
    11. sudo bash. Then export HADOOP_SECURE_DN_USER=raviprak . Then
    hadoop-daemon.sh start datanode



    CORE-SITE.XML
    ========================================
    <?xml version="1.0"?>
    <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>

    <!-- Put site-specific property overrides in this file. -->

    <configuration>
    <property>
    <name>fs.default.name</name>
    <value>hdfs://localhost:9001</value>
    </property>
    <property>
    <name>hadoop.security.authorization</name>
    <value>true</value>
    </property>
    <property>
    <name>hadoop.security.authentication</name>
    <value>kerberos</value>
    </property>
    <property>
    <name>dfs.namenode.kerberos.principal</name>
    <value>raviprak/localhost.localdomain</value>
    </property>
    <property>
    <name>dfs.datanode.kerberos.principal</name>
    <value>raviprak/localhost.localdomain</value>
    </property>
    <property>
    <name>dfs.secondary.namenode.kerberos.principal</name>
    <value>raviprak/localhost.localdomain</value>
    </property>
    </configuration>

    =========================================================



    HDFS-SITE.XML
    =========================================================
    <?xml version="1.0"?>
    <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>

    <!-- Put site-specific property overrides in this file. -->

    <configuration>
    <property>
    <name>dfs.replication</name>
    <value>1</value>
    </property>

    <property>
    <name>dfs.name.dir.restore</name>
    <value>false</value>
    </property>

    <property>
    <name>dfs.namenode.checkpoint.period</name>
    <value>10</value>
    </property>

    <property>
    <name>dfs.namenode.keytab.file</name>
    <value>/home/raviprak/raviprak.keytab</value>
    </property>

    <property>
    <name>dfs.secondary.namenode.keytab.file</name>
    <value>/home/raviprak/raviprak.keytab</value>
    </property>

    <property>
    <name>dfs.datanode.keytab.file</name>
    <value>/home/raviprak/raviprak.keytab</value>
    </property>

    <property>
    <name>dfs.datanode.address</name>
    <value>0.0.0.0:1004</value>
    </property>

    <property>
    <name>dfs.datanode.http.address</name>
    <value>0.0.0.0:1006</value>
    </property>

    <property>
    <name>dfs.namenode.kerberos.principal</name>
    <value>raviprak/localhost.localdomain@localdomain</value>
    </property>

    <property>
    <name>dfs.secondary.namenode.kerberos.principal</name>
    <value>raviprak/localhost.localdomain@localdomain</value>
    </property>

    <property>
    <name>dfs.datanode.kerberos.principal</name>
    <value>raviprak/localhost.localdomain@localdomain</value>
    </property>

    <property>
    <name>dfs.namenode.kerberos.https.principal</name>
    <value>raviprak/localhost.localdomain@localdomain</value>
    </property>

    <property>
    <name>dfs.secondary.namenode.kerberos.https.principal</name>
    <value>raviprak/localhost.localdomain@localdomain</value>
    </property>

    <property>
    <name>dfs.datanode.kerberos.https.principal</name>
    <value>raviprak/localhost.localdomain@localdomain</value>
    </property>

    </configuration>
    =========================================================

    On Tue, Aug 30, 2011 at 8:08 PM, Thomas Weise wrote:

    I'm configuring a local hadoop cluster in secure mode for
    development/experimental purposes on Ubuntu 11.04 with the hadoop-0.20.203.0
    distribution from apache mirror.

    I have the basic Kerberos setup working, can start namenode in secure mode
    and connect to it with hadoop fs -ls

    I'm not able to get the datanode start in secure mode - what do I have to
    do to make that happen?

    The error I get:

    11/08/30 18:01:57 INFO security.UserGroupInformation: Login successful for
    user hduser/hdev-vm@HADOOP.LOCALDOMAIN using keytab file
    /opt/hadoop/conf/nn.keytab
    11/08/30 18:01:57 ERROR datanode.DataNode: java.lang.RuntimeException:
    Cannot start secure cluster without privileged resources.
    at
    org.apache.hadoop.hdfs.server.datanode.DataNode.startDataNode(DataNode.java:293)
    at
    org.apache.hadoop.hdfs.server.datanode.DataNode.<init>(DataNode.java:268)
    at
    org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:1480)
    at
    org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:1419)
    at
    org.apache.hadoop.hdfs.server.datanode.DataNode.createDataNode(DataNode.java:1437)
    at
    org.apache.hadoop.hdfs.server.datanode.DataNode.secureMain(DataNode.java:1563)
    at
    org.apache.hadoop.hdfs.server.datanode.DataNode.main(DataNode.java:1573)

    11/08/30 18:01:57 INFO datanode.DataNode: SHUTDOWN_MSG:
    /************************************************************
    SHUTDOWN_MSG: Shutting down DataNode at hdev-vm/127.0.1.1

    I have not configured the system to use port numbers that require root
    (yet). All I want is the datanode to run in secure mode with kerberos
    authentication.

    Any pointers would be greatly appreciated!

    Thomas
  • Thomas Weise at Aug 31, 2011 at 8:33 pm
    Thanks Ravi. This has brought my local hadoop cluster to life!

    The two things I was missing:

    1) Have to use privileged ports

    <!-- secure setup requires privileged ports -->
    <property>
    <name>dfs.datanode.address</name>
    <value>0.0.0.0:1004</value>
    </property>
    <property>
    <name>dfs.datanode.http.address</name>
    <value>0.0.0.0:1006</value>
    </property>

    2) implied by 1) sudo required to launch datanode

    Clearly, this is geared towards the production system. For development, having the ability to run with Kerberos but w/o the need for privileged resources would be desirable.

    On Aug 30, 2011, at 9:00 PM, Ravi Prakash wrote:

    In short you MUST use priviledged resourced.

    In long:

    Here's what I did to setup a secure single node cluster. I'm sure there's
    other ways, but here's how I did it.

    1. Install krb5-server
    2. Setup the kerberos configuration (files attached).
    /var/kerberos/krb5kdc/kdc.conf and /etc/krb5.conf
    http://yahoo.github.com/hadoop-common/installing.html
    3. To clean up everything :
    http://mailman.mit.edu/pipermail/kerberos/2003-June/003312.html
    4. Create Kerberos database $ sudo kdb5_util create -s
    5. Start Kerberos $ sudo /etc/rc.d/init.d/kadmin start $ sudo
    /etc/rc.d/init.d/krb5kdc start
    6. Create principal raviprak/localhost.localdomain@localdomain
    http://web.mit.edu/kerberos/krb5-1.5/krb5-1.5.4/doc/krb5-admin/Adding-or-Modifying-Principals.html
    7. Create keytab fiie using “xst -k /home/raviprak/raviprak.keytab
    raviprak/localhost.localdomain@localdomain”
    8. Setup hdfs-site.xml and core-site.xml (files attached)
    9. sudo hostname localhost.localdomain
    10. hadoop-daemon.sh start namenode
    11. sudo bash. Then export HADOOP_SECURE_DN_USER=raviprak . Then
    hadoop-daemon.sh start datanode



    CORE-SITE.XML
    ========================================
    <?xml version="1.0"?>
    <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>

    <!-- Put site-specific property overrides in this file. -->

    <configuration>
    <property>
    <name>fs.default.name</name>
    <value>hdfs://localhost:9001</value>
    </property>
    <property>
    <name>hadoop.security.authorization</name>
    <value>true</value>
    </property>
    <property>
    <name>hadoop.security.authentication</name>
    <value>kerberos</value>
    </property>
    <property>
    <name>dfs.namenode.kerberos.principal</name>
    <value>raviprak/localhost.localdomain</value>
    </property>
    <property>
    <name>dfs.datanode.kerberos.principal</name>
    <value>raviprak/localhost.localdomain</value>
    </property>
    <property>
    <name>dfs.secondary.namenode.kerberos.principal</name>
    <value>raviprak/localhost.localdomain</value>
    </property>
    </configuration>

    =========================================================



    HDFS-SITE.XML
    =========================================================
    <?xml version="1.0"?>
    <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>

    <!-- Put site-specific property overrides in this file. -->

    <configuration>
    <property>
    <name>dfs.replication</name>
    <value>1</value>
    </property>

    <property>
    <name>dfs.name.dir.restore</name>
    <value>false</value>
    </property>

    <property>
    <name>dfs.namenode.checkpoint.period</name>
    <value>10</value>
    </property>

    <property>
    <name>dfs.namenode.keytab.file</name>
    <value>/home/raviprak/raviprak.keytab</value>
    </property>

    <property>
    <name>dfs.secondary.namenode.keytab.file</name>
    <value>/home/raviprak/raviprak.keytab</value>
    </property>

    <property>
    <name>dfs.datanode.keytab.file</name>
    <value>/home/raviprak/raviprak.keytab</value>
    </property>

    <property>
    <name>dfs.datanode.address</name>
    <value>0.0.0.0:1004</value>
    </property>

    <property>
    <name>dfs.datanode.http.address</name>
    <value>0.0.0.0:1006</value>
    </property>

    <property>
    <name>dfs.namenode.kerberos.principal</name>
    <value>raviprak/localhost.localdomain@localdomain</value>
    </property>

    <property>
    <name>dfs.secondary.namenode.kerberos.principal</name>
    <value>raviprak/localhost.localdomain@localdomain</value>
    </property>

    <property>
    <name>dfs.datanode.kerberos.principal</name>
    <value>raviprak/localhost.localdomain@localdomain</value>
    </property>

    <property>
    <name>dfs.namenode.kerberos.https.principal</name>
    <value>raviprak/localhost.localdomain@localdomain</value>
    </property>

    <property>
    <name>dfs.secondary.namenode.kerberos.https.principal</name>
    <value>raviprak/localhost.localdomain@localdomain</value>
    </property>

    <property>
    <name>dfs.datanode.kerberos.https.principal</name>
    <value>raviprak/localhost.localdomain@localdomain</value>
    </property>

    </configuration>
    =========================================================

    On Tue, Aug 30, 2011 at 8:08 PM, Thomas Weise wrote:

    I'm configuring a local hadoop cluster in secure mode for
    development/experimental purposes on Ubuntu 11.04 with the hadoop-0.20.203.0
    distribution from apache mirror.

    I have the basic Kerberos setup working, can start namenode in secure mode
    and connect to it with hadoop fs -ls

    I'm not able to get the datanode start in secure mode - what do I have to
    do to make that happen?

    The error I get:

    11/08/30 18:01:57 INFO security.UserGroupInformation: Login successful for
    user hduser/hdev-vm@HADOOP.LOCALDOMAIN using keytab file
    /opt/hadoop/conf/nn.keytab
    11/08/30 18:01:57 ERROR datanode.DataNode: java.lang.RuntimeException:
    Cannot start secure cluster without privileged resources.
    at
    org.apache.hadoop.hdfs.server.datanode.DataNode.startDataNode(DataNode.java:293)
    at
    org.apache.hadoop.hdfs.server.datanode.DataNode.<init>(DataNode.java:268)
    at
    org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:1480)
    at
    org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:1419)
    at
    org.apache.hadoop.hdfs.server.datanode.DataNode.createDataNode(DataNode.java:1437)
    at
    org.apache.hadoop.hdfs.server.datanode.DataNode.secureMain(DataNode.java:1563)
    at
    org.apache.hadoop.hdfs.server.datanode.DataNode.main(DataNode.java:1573)

    11/08/30 18:01:57 INFO datanode.DataNode: SHUTDOWN_MSG:
    /************************************************************
    SHUTDOWN_MSG: Shutting down DataNode at hdev-vm/127.0.1.1

    I have not configured the system to use port numbers that require root
    (yet). All I want is the datanode to run in secure mode with kerberos
    authentication.

    Any pointers would be greatly appreciated!

    Thomas

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-user @
categorieshadoop
postedAug 31, '11 at 1:09a
activeAug 31, '11 at 8:33p
posts3
users2
websitehadoop.apache.org...
irc#hadoop

2 users in discussion

Thomas Weise: 2 posts Ravi Prakash: 1 post

People

Translate

site design / logo © 2022 Grokbase