Grokbase › Groups › Hadoop › common-user › December 2010
FAQ
Is setting dfs.replication to 1 sufficient to stop replication? How do I verify that? I have a pseudo cluster running 0.21.0. It seems that the hdfs disk consumption triples the amount of data stored.

Thanks,
Jane

Search Discussions

  • Hemanth Yamijala at Dec 29, 2010 at 4:43 am ⇧
    Hi,
    On Wed, Dec 29, 2010 at 5:51 AM, Jane Chen wrote:
    Is setting dfs.replication to 1 sufficient to stop replication?  How do I verify that?  I have a pseudo cluster running 0.21.0.  It seems that the hdfs disk consumption triples the amount of data stored.
    Setting to 1 is sufficient to stop replication. Can you check if the
    web UI for NameNode has a way to show the replicas of blocks for a
    file ?
    Thanks,
    Jane


  • Jane Chen at Dec 29, 2010 at 5:09 pm ⇧
    You are right. There's only one replica. When does the deleted file space get reclaimed?

    --- On Tue, 12/28/10, Hemanth Yamijala wrote:
    From: Hemanth Yamijala <[email protected]>
    Subject: Re: HDFS disk consumption.
    To: [email protected]
    Date: Tuesday, December 28, 2010, 8:43 PM
    Hi,
    On Wed, Dec 29, 2010 at 5:51 AM, Jane Chen wrote:
    Is setting dfs.replication to 1 sufficient to stop
    replication?  How do I verify that?  I have a pseudo
    cluster running 0.21.0.  It seems that the hdfs disk
    consumption triples the amount of data stored.

    Setting to 1 is sufficient to stop replication. Can you
    check if the
    web UI for NameNode has a way to show the replicas of
    blocks for a
    file ?
    Thanks,
    Jane


  • Li ping at Dec 30, 2010 at 1:10 am ⇧
    If you remove the file from the command line. The file will be moved to
    trash folder(if you enable the feature).otherwise the file will be moved
    immediately.

    On Thu, Dec 30, 2010 at 1:09 AM, Jane Chen wrote:

    You are right. There's only one replica. When does the deleted file space
    get reclaimed?

    --- On Tue, 12/28/10, Hemanth Yamijala wrote:
    From: Hemanth Yamijala <[email protected]>
    Subject: Re: HDFS disk consumption.
    To: [email protected]
    Date: Tuesday, December 28, 2010, 8:43 PM
    Hi,

    On Wed, Dec 29, 2010 at 5:51 AM, Jane Chen <[email protected]>
    wrote:
    Is setting dfs.replication to 1 sufficient to stop
    replication? How do I verify that? I have a pseudo
    cluster running 0.21.0. It seems that the hdfs disk
    consumption triples the amount of data stored.

    Setting to 1 is sufficient to stop replication. Can you
    check if the
    web UI for NameNode has a way to show the replicas of
    blocks for a
    file ?
    Thanks,
    Jane




    --
    -----李平

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-user @
categorieshadoop
postedDec 29, '10 at 12:22a
activeDec 30, '10 at 1:10a
posts4
users3
websitehadoop.apache.org...
irc#hadoop

3 users in discussion

Jane Chen: 2 posts Li ping: 1 post Hemanth Yamijala: 1 post

Content

People

Support

Translate

site design / logo © 2023 Grokbase