FAQ
Hi all,

We are facing issues while porting some logs to HDFS. The way we are doing it is using a simple java code which tries to read the file and writes to HDFS using OutputStream. It was working perfectly fine and recently, we are getting below error messages once in a while and when we try to read that data for that data we are getting error message as "Could not obtain block" and the jobs are failing.

Can some one tell me what would be the issue?

The error message while writing to HDFS is:

WARN dfs.DFSClient: DFSOutputStream ResponseProcessor
exception for block blk_-1005367228931083977_10012402java.io.IOException:
Bad response 1 for block blk_-1005367228931083977_10012402 from datanode
xxx.xxx.xxx.88:50010
at
org.apache.hadoop.dfs.DFSClient$DFSOutputStream$ResponseProcessor.run(DFSCli
ent.java:2076)

09/06/08 14:26:41 WARN dfs.DFSClient: Error Recovery for block
blk_-1005367228931083977_10012402 bad datanode[1] xxx.xxx.xxx.88:50010
09/06/08 14:26:41 WARN dfs.DFSClient: Error Recovery for block
blk_-1005367228931083977_10012402 in pipeline xxx.xxx.xxx.79:50010,
xxx.xxx.xxx.88:50010, xxx.xxx.xxx.68:50010: bad datanode
xxx.xxx.xxx.88:50010
09/06/08 14:26:42 WARN dfs.DFSClient: DFSOutputStream ResponseProcessor
exception for block blk_-1005367228931083977_10012456java.io.IOException:
Bad response 1 for block blk_-1005367228931083977_10012456 from datanode
xxx.xxx.xxx.68:50010
at
org.apache.hadoop.dfs.DFSClient$DFSOutputStream$ResponseProcessor.run(DFSCli
ent.java:2076)

09/06/08 14:26:42 WARN dfs.DFSClient: Error Recovery for block
blk_-1005367228931083977_10012456 bad datanode[1] xxx.xxx.xxx.68:50010
09/06/08 14:26:42 WARN dfs.DFSClient: Error Recovery for block
blk_-1005367228931083977_10012456 in pipeline xxx.xxx.xxx.79:50010,
xxx.xxx.xxx.68:50010: bad datanode xxx.xxx.xxx.68:50010
09/06/08 14:26:43 WARN dfs.DFSClient: DFSOutputStream ResponseProcessor
exception for block blk_-1005367228931083977_10012457java.io.EOFException
at java.io.DataInputStream.readFully(DataInputStream.java:180)
at java.io.DataInputStream.readLong(DataInputStream.java:399)
at
org.apache.hadoop.dfs.DFSClient$DFSOutputStream$ResponseProcessor.run(DFSCli
ent.java:2052)

09/06/08 14:26:43 WARN dfs.DFSClient: Error Recovery for block
blk_-1005367228931083977_10012457 bad datanode[0] xxx.xxx.xxx.79:50010
IOException - java.io.IOException: All datanodes xxx.xxx.xxx.79:50010 are
bad. Aborting... while writing -

Thanks
Pallavi

Search Discussions

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-user @
categorieshadoop
postedJun 17, '09 at 7:36a
activeJun 17, '09 at 7:36a
posts1
users1
websitehadoop.apache.org...
irc#hadoop

1 user in discussion

Pallavi Palleti: 1 post

People

Translate

site design / logo © 2022 Grokbase