FAQ
I will put the answer and the question in reverse order.

ANSWER: Modify dfs.datanode.du.reserved to be 100MB instead of
10GB, for each datanode. Search in the cloudera GUI, and you will
find this value which is usually set to 10GB. Change each datanode,
save, and restart.

And to Cloudera: please make this change 'dynamically' during
installation, or something, for goodness sakes. :-) what a
mess.

QUESTION: Why do I get all these terrible errors when I try to
test on Amazon EC2 micro instances?

It is not obvious what is causing them... Everything appears to be
running ok.

12/06/18 11:46:27 WARN hdfs.DFSClient: DataStreamer Exception:
org.apache.hadoop.ipc.RemoteException: java.io.IOException: File /tmp/
test.txt

could only be replicated to 0 nodes, instead of 1


12/06/18 11:46:27 WARN hdfs.DFSClient: Error Recovery for block null
bad datanode[0] nodes == null
12/06/18 11:46:27 WARN hdfs.DFSClient: Could not get block locations.
Source file "/tmp/test.txt" - Aborting...
put: java.io.IOException: File /tmp/test.txt could only be replicated
to 0 nodes, instead of 1
12/06/18 11:46:27 ERROR hdfs.DFSClient: Exception closing file /tmp/
test.txt : org.apache.hadoop.ipc.RemoteException: java.io.IOException:
File

/tmp/test.txt could only be replicated to 0 nodes, instead of 1

2012-06-18 11:44:27,757 ERROR
org.apache.hadoop.security.UserGroupInformation:
PriviledgedActionException as:hbase (auth:SIMPLE)

cause:java.io.IOException: File /hbase/hbase.version could only be
replicated to 0 nodes, instead of 1
error: java.io.IOException: File /hbase/hbase.version could only be
replicated to 0 nodes, instead of 1

Search Discussions

  • Sam Darwin at Jun 18, 2012 at 5:15 pm
    BTW, in relation to the last post, the clue I found which led me to
    the answer on this very group was:

    Configured Capacity : 0

    And this was because there wasn't enough disk space on the EC2
    instance and therefore the datanode wasn't allocating itself any space
    at all.
  • Vinithra Varadharajan at Jun 20, 2012 at 6:06 pm
    Hi Sam,

    Apologies for the inconvenience that this caused you. This will be fixed in
    the next release of CM.

    -Vinithra
    On Mon, Jun 18, 2012 at 10:11 AM, Sam Darwin wrote:

    I will put the answer and the question in reverse order.

    ANSWER: Modify dfs.datanode.du.reserved to be 100MB instead of
    10GB, for each datanode. Search in the cloudera GUI, and you will
    find this value which is usually set to 10GB. Change each datanode,
    save, and restart.

    And to Cloudera: please make this change 'dynamically' during
    installation, or something, for goodness sakes. :-) what a
    mess.

    QUESTION: Why do I get all these terrible errors when I try to
    test on Amazon EC2 micro instances?

    It is not obvious what is causing them... Everything appears to be
    running ok.

    12/06/18 11:46:27 WARN hdfs.DFSClient: DataStreamer Exception:
    org.apache.hadoop.ipc.RemoteException: java.io.IOException: File /tmp/
    test.txt

    could only be replicated to 0 nodes, instead of 1


    12/06/18 11:46:27 WARN hdfs.DFSClient: Error Recovery for block null
    bad datanode[0] nodes == null
    12/06/18 11:46:27 WARN hdfs.DFSClient: Could not get block locations.
    Source file "/tmp/test.txt" - Aborting...
    put: java.io.IOException: File /tmp/test.txt could only be replicated
    to 0 nodes, instead of 1
    12/06/18 11:46:27 ERROR hdfs.DFSClient: Exception closing file /tmp/
    test.txt : org.apache.hadoop.ipc.RemoteException: java.io.IOException:
    File

    /tmp/test.txt could only be replicated to 0 nodes, instead of 1

    2012-06-18 11:44:27,757 ERROR
    org.apache.hadoop.security.UserGroupInformation:
    PriviledgedActionException as:hbase (auth:SIMPLE)

    cause:java.io.IOException: File /hbase/hbase.version could only be
    replicated to 0 nodes, instead of 1
    error: java.io.IOException: File /hbase/hbase.version could only be
    replicated to 0 nodes, instead of 1

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupscm-users @
categorieshadoop
postedJun 18, '12 at 5:12p
activeJun 20, '12 at 6:06p
posts3
users2
websitecloudera.com
irc#hadoop

People

Translate

site design / logo © 2022 Grokbase