In short you MUST use priviledged resourced.
In long:
Here's what I did to setup a secure single node cluster. I'm sure there's
other ways, but here's how I did it.
1. Install krb5-server
2. Setup the kerberos configuration (files attached).
/var/kerberos/krb5kdc/kdc.conf and /etc/krb5.conf
http://yahoo.github.com/hadoop-common/installing.html3. To clean up everything :
http://mailman.mit.edu/pipermail/kerberos/2003-June/003312.html4. Create Kerberos database $ sudo kdb5_util create -s
5. Start Kerberos $ sudo /etc/rc.d/init.d/kadmin start $ sudo
/etc/rc.d/init.d/krb5kdc start
6. Create principal raviprak/localhost.localdomain@localdomain
http://web.mit.edu/kerberos/krb5-1.5/krb5-1.5.4/doc/krb5-admin/Adding-or-Modifying-Principals.html7. Create keytab fiie using “xst -k /home/raviprak/raviprak.keytab
raviprak/localhost.localdomain@localdomain”
8. Setup hdfs-site.xml and core-site.xml (files attached)
9. sudo hostname localhost.localdomain
10. hadoop-daemon.sh start namenode
11. sudo bash. Then export HADOOP_SECURE_DN_USER=raviprak . Then
hadoop-daemon.sh start datanode
CORE-SITE.XML
========================================
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!-- Put site-specific property overrides in this file. -->
<configuration>
<property>
<name>fs.default.name</name>
<value>hdfs://localhost:9001</value>
</property>
<property>
<name>hadoop.security.authorization</name>
<value>true</value>
</property>
<property>
<name>hadoop.security.authentication</name>
<value>kerberos</value>
</property>
<property>
<name>dfs.namenode.kerberos.principal</name>
<value>raviprak/localhost.localdomain</value>
</property>
<property>
<name>dfs.datanode.kerberos.principal</name>
<value>raviprak/localhost.localdomain</value>
</property>
<property>
<name>dfs.secondary.namenode.kerberos.principal</name>
<value>raviprak/localhost.localdomain</value>
</property>
</configuration>
=========================================================
HDFS-SITE.XML
=========================================================
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!-- Put site-specific property overrides in this file. -->
<configuration>
<property>
<name>dfs.replication</name>
<value>1</value>
</property>
<property>
<name>dfs.name.dir.restore</name>
<value>false</value>
</property>
<property>
<name>dfs.namenode.checkpoint.period</name>
<value>10</value>
</property>
<property>
<name>dfs.namenode.keytab.file</name>
<value>/home/raviprak/raviprak.keytab</value>
</property>
<property>
<name>dfs.secondary.namenode.keytab.file</name>
<value>/home/raviprak/raviprak.keytab</value>
</property>
<property>
<name>dfs.datanode.keytab.file</name>
<value>/home/raviprak/raviprak.keytab</value>
</property>
<property>
<name>dfs.datanode.address</name>
<value>0.0.0.0:1004</value>
</property>
<property>
<name>dfs.datanode.http.address</name>
<value>0.0.0.0:1006</value>
</property>
<property>
<name>dfs.namenode.kerberos.principal</name>
<value>raviprak/localhost.localdomain@localdomain</value>
</property>
<property>
<name>dfs.secondary.namenode.kerberos.principal</name>
<value>raviprak/localhost.localdomain@localdomain</value>
</property>
<property>
<name>dfs.datanode.kerberos.principal</name>
<value>raviprak/localhost.localdomain@localdomain</value>
</property>
<property>
<name>dfs.namenode.kerberos.https.principal</name>
<value>raviprak/localhost.localdomain@localdomain</value>
</property>
<property>
<name>dfs.secondary.namenode.kerberos.https.principal</name>
<value>raviprak/localhost.localdomain@localdomain</value>
</property>
<property>
<name>dfs.datanode.kerberos.https.principal</name>
<value>raviprak/localhost.localdomain@localdomain</value>
</property>
</configuration>
=========================================================
On Tue, Aug 30, 2011 at 8:08 PM, Thomas Weise wrote:
I'm configuring a local hadoop cluster in secure mode for
development/experimental purposes on Ubuntu 11.04 with the hadoop-0.20.203.0
distribution from apache mirror.
I have the basic Kerberos setup working, can start namenode in secure mode
and connect to it with hadoop fs -ls
I'm not able to get the datanode start in secure mode - what do I have to
do to make that happen?
The error I get:
11/08/30 18:01:57 INFO security.UserGroupInformation: Login successful for
user hduser/hdev-vm@HADOOP.LOCALDOMAIN using keytab file
/opt/hadoop/conf/nn.keytab
11/08/30 18:01:57 ERROR datanode.DataNode: java.lang.RuntimeException:
Cannot start secure cluster without privileged resources.
at
org.apache.hadoop.hdfs.server.datanode.DataNode.startDataNode(DataNode.java:293)
at
org.apache.hadoop.hdfs.server.datanode.DataNode.<init>(DataNode.java:268)
at
org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:1480)
at
org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:1419)
at
org.apache.hadoop.hdfs.server.datanode.DataNode.createDataNode(DataNode.java:1437)
at
org.apache.hadoop.hdfs.server.datanode.DataNode.secureMain(DataNode.java:1563)
at
org.apache.hadoop.hdfs.server.datanode.DataNode.main(DataNode.java:1573)
11/08/30 18:01:57 INFO datanode.DataNode: SHUTDOWN_MSG:
/************************************************************
SHUTDOWN_MSG: Shutting down DataNode at hdev-vm/127.0.1.1
I have not configured the system to use port numbers that require root
(yet). All I want is the datanode to run in secure mode with kerberos
authentication.
Any pointers would be greatly appreciated!
Thomas
