Hi -
I’m seeing the following exception while trying to start MiniDFSCluster in some environment. The stack trace is:
Starting DataNode 0 with dfs.data.dir: build/test/data/dfs/data/data1,build/test/data/dfs/data/data2
java.lang.NullPointerException
at org.apache.hadoop.hdfs.MiniDFSCluster.startDataNodes(MiniDFSCluster.java:413)
at org.apache.hadoop.hdfs.MiniDFSCluster.(MiniDFSCluster.java:122)
at com.yahoo.ads.ngdstone.tpbdm.BDMTestCase.oneTimeSetUp(BDMTestCase.java:69)
My code is:
System.setProperty("hadoop.log.dir", "/tmp");
System.setProperty("dfs.permissions.supergroup", "su");
conf = new Configuration();
dfsCluster = new MiniDFSCluster(conf, 1, true, null); // <== Line 69 in BDMTestCase.java
fs = dfsCluster.getFileSystem();
...
Line 413 in MiniDFSCluster is:
String ipAddr = dn.getSelfAddr().getAddress().getHostAddress();
So I’m guessing it has something to do with the network setup?? - I see just above in MiniDFSCluster:
conf.set("dfs.datanode.address", "127.0.0.1:0");
conf.set("dfs.datanode.http.address", "127.0.0.1:0");
conf.set("dfs.datanode.ipc.address", "127.0.0.1:0");
Do I need to have those ports available/enabled in my environment? Is that the problem here?
Thanks!
Frank
