FAQ
I see createBlockException and Abandoning block quite often
When I check the datanode, they are running. I can browse file system
from that datanode:50075
However, I also notice tht a du forked off from the DN. This 'du' run
anywhere from 6mins to 30 mins.

During this time no logs are generated . DN appears in S1 state and the
'du' in D state.

Is it possible tht jvm has bug or hdd is bad.
I m using "/usr/java/jdk1.6.0_07/bin/java" and planing to move onto 11

However, I start noticing this after DFS is 50% (on avg) full

Pl help me with some pointers

Hadoop version : .18.1

-Sagar

Search Discussions

  • Brian Bockelman at Dec 18, 2008 at 2:36 am
    Hey Sagar,

    If the 'du' is in the "D" state, then that probably means bad things
    for your hardware.

    I recommend looking in dmesg and /var/log/messages for anything
    interesting, as well as perform a hard-drive diagnostic test (may be
    as simple as a SMART tests) to see if there's an issue.

    I can't say for sure, but the 'du' is probably not hanging the
    Datanode; it's probably a symptom of larger problems.

    Brian
    On Dec 17, 2008, at 8:29 PM, Sagar Naik wrote:


    I see createBlockException and Abandoning block quite often
    When I check the datanode, they are running. I can browse file
    system from that datanode:50075
    However, I also notice tht a du forked off from the DN. This 'du'
    run anywhere from 6mins to 30 mins.

    During this time no logs are generated . DN appears in S1 state and
    the 'du' in D state.

    Is it possible tht jvm has bug or hdd is bad.
    I m using "/usr/java/jdk1.6.0_07/bin/java" and planing to move onto 11

    However, I start noticing this after DFS is 50% (on avg) full

    Pl help me with some pointers

    Hadoop version : .18.1

    -Sagar



  • Sagar Naik at Dec 18, 2008 at 2:46 am

    Brian Bockelman wrote:
    Hey Sagar,

    If the 'du' is in the "D" state, then that probably means bad things
    for your hardware.

    I recommend looking in dmesg and /var/log/messages for anything
    interesting, as well as perform a hard-drive diagnostic test (may be
    as simple as a SMART tests) to see if there's an issue.

    I can't say for sure, but the 'du' is probably not hanging the
    Datanode; it's probably a symptom of larger problems.
    Thanks Brian
    I will start SMART tests
    Pl tell me what direction I should look in case of "larger problems".

    Brian
    On Dec 17, 2008, at 8:29 PM, Sagar Naik wrote:


    I see createBlockException and Abandoning block quite often
    When I check the datanode, they are running. I can browse file system
    from that datanode:50075
    However, I also notice tht a du forked off from the DN. This 'du'
    run anywhere from 6mins to 30 mins.

    During this time no logs are generated . DN appears in S1 state and
    the 'du' in D state.

    Is it possible tht jvm has bug or hdd is bad.
    I m using "/usr/java/jdk1.6.0_07/bin/java" and planing to move onto 11

    However, I start noticing this after DFS is 50% (on avg) full

    Pl help me with some pointers

    Hadoop version : .18.1

    -Sagar



Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-user @
categorieshadoop
postedDec 18, '08 at 2:30a
activeDec 18, '08 at 2:46a
posts3
users2
websitehadoop.apache.org...
irc#hadoop

2 users in discussion

Sagar Naik: 2 posts Brian Bockelman: 1 post

People

Translate

site design / logo © 2021 Grokbase