Hi all
I met a problem when I try to balance certain hdfs directory among the
clusters. For example, I have a directory "/user/xxx/", and there 100
blocks. I want to balance them among my 5 nodes clusters. Each node has 40
blocks (2 replicas). The problem is about transfer block from one datanode
to another. Actually, I followed the balancer's method. However, it always
waits for the response of destination datanode and halt. I attached the
code:
.................................................
Socket sock = new Socket();
DataOutputStream out = null;
DataInputStream in = null;
try{
sock.connect(NetUtils.createSocketAddr(
target.getName()), HdfsConstants.READ_TIMEOUT);
sock.setKeepAlive(true);
System.out.println(sock.isConnected());
out = new DataOutputStream( new BufferedOutputStream(
sock.getOutputStream(), FSConstants.BUFFER_SIZE));
out.writeShort(DataTransferProtocol.DATA_TRANSFER_VERSION);
out.writeByte(DataTransferProtocol.OP_REPLACE_BLOCK);
out.writeLong(block2move.getBlockId());
out.writeLong(block2move.getGenerationStamp());
Text.writeString(out, source.getStorageID());
System.out.println("Ready to move");
source.write(out);
System.out.println("Write to output Stream");
out.flush();
System.out.println("out has been flushed!");
in = new DataInputStream( new BufferedInputStream(
sock.getInputStream(), FSConstants.BUFFER_SIZE));
It stop here and wait for response.
short status = in.readShort();
System.out.println("Got the response from input stream!"+status);
if (status != DataTransferProtocol.OP_STATUS_SUCCESS) {
throw new IOException("block move is failed\t"+status);
}
} catch (IOException e) {
LOG.warn("Error moving block "+block2move.getBlockId()+
" from " + source.getName() + " to " +
target.getName() + " through " +
source.getName() +
": "+e.toString());
} finally {
IOUtils.closeStream(out);
IOUtils.closeStream(in);
IOUtils.closeSocket(sock);
}
..........................................
Any reply will be appreciated. Thank you in advance!
Chen
