Hi,
I am evaluating Hadoop's Portability Across Heterogeneous Hardware and
Software Platforms. For this I am trying to setup a grid (Hadoop 0.17)
having Linux ( RHEL5 / FC 9), Solaris (SunOS 5) and AIX (5.3). I was able to
setup a grid with 10 Linux machines and run some basic grid jobs on it. I
was also able to setup and start off a standalone grid in an AIX machine.
But when I try to copy data into this (AIX) grid, I get the following error
–
*$ /opt/hadoop/bin/hadoop dfs -ls*
*ls: Cannot access .: No such file or directory.*
*$ /opt/hadoop/bin/hadoop dfs -copyFromLocal option_100_data.csv
option_100_data.csv*
*08/08/13 02:36:50 INFO dfs.DFSClient:
org.apache.hadoop.ipc.RemoteException: java.io.IOException: File
/user/hadoop/option_100_data.csv could only be replicated to 0 nodes,
instead of 1*
* at
org.apache.hadoop.dfs.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1145)
*
* at org.apache.hadoop.dfs.NameNode.addBlock(NameNode.java:300)*
* at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)*
* at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:64)
*
* at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
*
* at java.lang.reflect.Method.invoke(Method.java:615)*
* at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:446)*
* at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)*
* *
* at org.apache.hadoop.ipc.Client.call(Client.java:557)*
* at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:212)*
* at org.apache.hadoop.dfs.$Proxy0.addBlock(Unknown Source)*
* at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)*
* at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:64)
*
* at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
*
* at java.lang.reflect.Method.invoke(Method.java:615)*
* at
org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
*
* at
org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
*
* at org.apache.hadoop.dfs.$Proxy0.addBlock(Unknown Source)*
* at
org.apache.hadoop.dfs.DFSClient$DFSOutputStream.locateFollowingBlock(DFSClient.java:2334)
*
* at
org.apache.hadoop.dfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:2219)
*
* at
org.apache.hadoop.dfs.DFSClient$DFSOutputStream.access$1700(DFSClient.java:1702)
*
* at
org.apache.hadoop.dfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:1842)
*
*
*
*08/08/13 02:36:50 WARN dfs.DFSClient: NotReplicatedYetException sleeping
/user/hadoop/option_100_data.csv retries left 4*
*08/08/13 02:36:51 INFO dfs.DFSClient:
org.apache.hadoop.ipc.RemoteException: java.io.IOException: File
/user/hadoop/option_100_data.csv could only be replicated to 0 nodes,
instead of 1*
* at
org.apache.hadoop.dfs.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1145)
*
* …………………………………………………………………………………………………………………………………………………………………………*
*…………………………………………………………………………………………………………………………………………………………………………*
* …………………………………………………………………………………………………………………………………………………………………………*
*08/08/13 02:36:57 WARN dfs.DFSClient: Error Recovery for block null bad
datanode[0]*
*$*
This is the only error I am seeing in the log files. My first doubt was if
the issue was due to IBM JDK. To verify this I tried to run Hadoop using IBM
JDK in Linux (FC 9) machine and it worked perfectly.
It would be a great help if someone could give me some pointers on what I
can try to solve / debug this error.
Thanks,
Arun
