FAQ
I am trying to resolve an IOException error. I have a basic setup and shortly after running start-dfs.sh I get a:

error: java.io.IOException: File /tmp/hadoop-root/mapred/system/jobtracker.info could only be replicated to 0 nodes, instead of 1
java.io.IOException: File /tmp/hadoop-root/mapred/system/jobtracker.info could only be replicated to 0 nodes, instead of 1

Any pointers how to resolve this? Thanks!

Search Discussions

  • Boris Shkolnik at Nov 9, 2009 at 10:03 pm
    Make sure you have at least one datanode running.
    Look at the data node log file. (logs/*-datanode-*.log)

    Boris.

    On 11/9/09 7:15 AM, "Raymond Jennings III" wrote:

    I am trying to resolve an IOException error. I have a basic setup and shortly
    after running start-dfs.sh I get a:

    error: java.io.IOException: File
    /tmp/hadoop-root/mapred/system/jobtracker.info could only be replicated to 0
    nodes, instead of 1
    java.io.IOException: File /tmp/hadoop-root/mapred/system/jobtracker.info could
    only be replicated to 0 nodes, instead of 1

    Any pointers how to resolve this? Thanks!


  • Raymond Jennings III at Nov 10, 2009 at 4:02 pm
    On the actual datanodes I see the following exception: I am not sure what the namespaceID is or how to sync them. Thanks for any advice!



    /************************************************************
    STARTUP_MSG: Starting DataNode
    STARTUP_MSG: host = pingo-3.poly.edu/128.238.55.33
    STARTUP_MSG: args = []
    STARTUP_MSG: version = 0.20.1
    STARTUP_MSG: build = http://svn.apache.org/repos/asf/hadoop/common/tags/release-0.20.1-rc1 -r 810220; compiled by 'oom' on Tue Sep 1 20:55:56 UTC 2009
    ************************************************************/
    2009-11-09 09:57:45,328 ERROR org.apache.hadoop.hdfs.server.datanode.DataNode: java.io.IOException: Incompatible namespaceIDs in /tmp/hadoop-root/dfs/data: namenode namespaceID = 1016244663; datanode namespaceID = 1687029285
    at org.apache.hadoop.hdfs.server.datanode.DataStorage.doTransition(DataStorage.java:233)
    at org.apache.hadoop.hdfs.server.datanode.DataStorage.recoverTransitionRead(DataStorage.java:148)
    at org.apache.hadoop.hdfs.server.datanode.DataNode.startDataNode(DataNode.java:298)
    at org.apache.hadoop.hdfs.server.datanode.DataNode.(DataNode.java:1283)
    at org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:1238)
    at org.apache.hadoop.hdfs.server.datanode.DataNode.createDataNode(DataNode.java:1246)
    at org.apache.hadoop.hdfs.server.datanode.DataNode.main(DataNode.java:1368)


    --- On Mon, 11/9/09, Boris Shkolnik wrote:
    From: Boris Shkolnik <borya@yahoo-inc.com>
    Subject: Re: newbie question - error with replication
    To: common-user@hadoop.apache.org
    Date: Monday, November 9, 2009, 5:02 PM
    Make sure you have at least one
    datanode running.
    Look at the data node log file. (logs/*-datanode-*.log)

    Boris.

    On 11/9/09 7:15 AM, "Raymond Jennings III" wrote:

    I am trying to resolve an IOException error.  I
    have a basic setup and shortly
    after running start-dfs.sh I get a:

    error: java.io.IOException: File
    /tmp/hadoop-root/mapred/system/jobtracker.info could
    only be replicated to 0
    nodes, instead of 1
    java.io.IOException: File
    /tmp/hadoop-root/mapred/system/jobtracker.info could
    only be replicated to 0 nodes, instead of 1

    Any pointers how to resolve this?  Thanks!


  • Edmund Kohlwey at Nov 10, 2009 at 6:47 pm
    Hi Ray,
    You'll probably find that even though the name node starts, it doesn't
    have any data nodes and is completely empty.

    Whenever hadoop creates a new filesystem, it assigns a large random
    number to it to prevent you from mixing datanodes from different
    filesystems on accident. When you reformat the name node its FS has one
    ID, but your data nodes still have chunks of the old FS with a different
    ID and so will refuse to connect to the namenode. You need to make sure
    these are cleaned up before reformatting. You can do it just by deleting
    the data node directory, although there's probably a more "official" way
    to do it.

    On 11/10/09 11:01 AM, Raymond Jennings III wrote:
    On the actual datanodes I see the following exception: I am not sure what the namespaceID is or how to sync them. Thanks for any advice!



    /************************************************************
    STARTUP_MSG: Starting DataNode
    STARTUP_MSG: host = pingo-3.poly.edu/128.238.55.33
    STARTUP_MSG: args = []
    STARTUP_MSG: version = 0.20.1
    STARTUP_MSG: build = http://svn.apache.org/repos/asf/hadoop/common/tags/release-0.20.1-rc1 -r 810220; compiled by 'oom' on Tue Sep 1 20:55:56 UTC 2009
    ************************************************************/
    2009-11-09 09:57:45,328 ERROR org.apache.hadoop.hdfs.server.datanode.DataNode: java.io.IOException: Incompatible namespaceIDs in /tmp/hadoop-root/dfs/data: namenode namespaceID = 1016244663; datanode namespaceID = 1687029285
    at org.apache.hadoop.hdfs.server.datanode.DataStorage.doTransition(DataStorage.java:233)
    at org.apache.hadoop.hdfs.server.datanode.DataStorage.recoverTransitionRead(DataStorage.java:148)
    at org.apache.hadoop.hdfs.server.datanode.DataNode.startDataNode(DataNode.java:298)
    at org.apache.hadoop.hdfs.server.datanode.DataNode.<init>(DataNode.java:216)
    at org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:1283)
    at org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:1238)
    at org.apache.hadoop.hdfs.server.datanode.DataNode.createDataNode(DataNode.java:1246)
    at org.apache.hadoop.hdfs.server.datanode.DataNode.main(DataNode.java:1368)


    --- On Mon, 11/9/09, Boris Shkolnikwrote:

    From: Boris Shkolnik<borya@yahoo-inc.com>
    Subject: Re: newbie question - error with replication
    To: common-user@hadoop.apache.org
    Date: Monday, November 9, 2009, 5:02 PM
    Make sure you have at least one
    datanode running.
    Look at the data node log file. (logs/*-datanode-*.log)

    Boris.


    On 11/9/09 7:15 AM, "Raymond Jennings III"<raymondjiii@yahoo.com>
    wrote:

    I am trying to resolve an IOException error. I
    have a basic setup and shortly
    after running start-dfs.sh I get a:

    error: java.io.IOException: File
    /tmp/hadoop-root/mapred/system/jobtracker.info could
    only be replicated to 0
    nodes, instead of 1
    java.io.IOException: File
    /tmp/hadoop-root/mapred/system/jobtracker.info could
    only be replicated to 0 nodes, instead of 1

    Any pointers how to resolve this? Thanks!



  • Raymond Jennings III at Nov 10, 2009 at 8:13 pm
    Thanks!!! That worked! I guess I can edit the number on the datanodes as well but if there is an even more "official" way to resolve this I would be interested in hearing about it.

    --- On Tue, 11/10/09, Edmund Kohlwey wrote:
    From: Edmund Kohlwey <ekohlwey@gmail.com>
    Subject: Re: Error with replication and namespaceID
    To: common-user@hadoop.apache.org
    Date: Tuesday, November 10, 2009, 1:46 PM
    Hi Ray,
    You'll probably find that even though the name node starts,
    it doesn't
    have any data nodes and is completely empty.

    Whenever hadoop creates a new filesystem, it assigns a
    large random
    number to it to prevent you from mixing datanodes from
    different
    filesystems on accident. When you reformat the name node
    its FS has one
    ID, but your data nodes still have chunks of the old FS
    with a different
    ID and so will refuse to connect to the namenode. You need
    to make sure
    these are cleaned up before reformatting. You can do it
    just by deleting
    the data node directory, although there's probably a more
    "official" way
    to do it.

    On 11/10/09 11:01 AM, Raymond Jennings III wrote:
    On the actual datanodes I see the following
    exception:  I am not sure what the namespaceID is or
    how to sync them.  Thanks for any advice!


    /************************************************************
    STARTUP_MSG: Starting DataNode
    STARTUP_MSG:   host =
    pingo-3.poly.edu/128.238.55.33
    STARTUP_MSG:   args = []
    STARTUP_MSG:   version = 0.20.1
    STARTUP_MSG:   build = http://svn.apache.org/repos/asf/hadoop/common/tags/release-0.20.1-rc1
    -r 810220; compiled by 'oom' on Tue Sep  1 20:55:56 UTC
    2009

    ************************************************************/
    2009-11-09 09:57:45,328 ERROR
    org.apache.hadoop.hdfs.server.datanode.DataNode:
    java.io.IOException: Incompatible namespaceIDs in
    /tmp/hadoop-root/dfs/data: namenode namespaceID =
    1016244663; datanode namespaceID = 1687029285
    at
    org.apache.hadoop.hdfs.server.datanode.DataStorage.doTransition(DataStorage.java:233)
    at
    org.apache.hadoop.hdfs.server.datanode.DataStorage.recoverTransitionRead(DataStorage.java:148)
    at
    org.apache.hadoop.hdfs.server.datanode.DataNode.startDataNode(DataNode.java:298)
    at
    org.apache.hadoop.hdfs.server.datanode.DataNode.<init>(DataNode.java:216)
    at
    org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:1283)
    at
    org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:1238)
    at
    org.apache.hadoop.hdfs.server.datanode.DataNode.createDataNode(DataNode.java:1246)
    at
    org.apache.hadoop.hdfs.server.datanode.DataNode.main(DataNode.java:1368)

    --- On Mon, 11/9/09, Boris Shkolnikwrote:

    From: Boris Shkolnik<borya@yahoo-inc.com>
    Subject: Re: newbie question - error with
    replication
    To: common-user@hadoop.apache.org
    Date: Monday, November 9, 2009, 5:02 PM
    Make sure you have at least one
    datanode running.
    Look at the data node log file.
    (logs/*-datanode-*.log)
    Boris.


    On 11/9/09 7:15 AM, "Raymond Jennings III"<raymondjiii@yahoo.com>
    wrote:

    I am trying to resolve an IOException
    error.  I
    have a basic setup and shortly
    after running start-dfs.sh I get a:

    error: java.io.IOException: File
    /tmp/hadoop-root/mapred/system/jobtracker.info
    could
    only be replicated to 0
    nodes, instead of 1
    java.io.IOException: File
    /tmp/hadoop-root/mapred/system/jobtracker.info
    could
    only be replicated to 0 nodes, instead of 1

    Any pointers how to resolve this?
    Thanks!


Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-user @
categorieshadoop
postedNov 9, '09 at 3:15p
activeNov 10, '09 at 8:13p
posts5
users3
websitehadoop.apache.org...
irc#hadoop

People

Translate

site design / logo © 2022 Grokbase