FAQ
Can't find much information regarding this. How to arrive at a good
value...does changing it effect the write throughput of a cluster.

Any info/pointers would be highly appreciated.

--
- Inder
"You are average of the 5 people you spend the most time with"

--

Search Discussions

  • Muhamed Ghareeb at Dec 4, 2012 at 11:32 am
    take a look for the below article
    http://blog.cloudera.com/blog/2009/03/configuration-parameters-what-can-you-just-ignore/



    On Tuesday, December 4, 2012 12:56:49 PM UTC+2, inder singh wrote:

    Can't find much information regarding this. How to arrive at a good
    value...does changing it effect the write throughput of a cluster.

    Any info/pointers would be highly appreciated.

    --
    - Inder
    "You are average of the 5 people you spend the most time with"
    --
  • Inder Pall at Dec 4, 2012 at 11:50 am
    Thx Muhammed saw this one however wanted more depth w.r.t this and
    dfs.namnode.handler.count as what are side-effects of a low setting...how
    do you know if the setting is good enough and stuff like that.

    - inder


    On Tue, Dec 4, 2012 at 5:02 PM, Muhamed Ghareeb
    wrote:
    take a look for the below article

    http://blog.cloudera.com/blog/2009/03/configuration-parameters-what-can-you-just-ignore/



    On Tuesday, December 4, 2012 12:56:49 PM UTC+2, inder singh wrote:

    Can't find much information regarding this. How to arrive at a good
    value...does changing it effect the write throughput of a cluster.

    Any info/pointers would be highly appreciated.

    --
    - Inder
    "You are average of the 5 people you spend the most time with"

    --



    --
    - Inder
    "You are average of the 5 people you spend the most time with"

    --
  • Aaron T. Myers at Dec 4, 2012 at 6:51 pm
    The DN handler count almost never needs to be increased. It's the number of
    RPC handlers, and very few things actually make RPC calls to DNs. Actual
    reads/writes to DNs are done over the DataTransferProtocol, which doesn't
    use Hadoop RPC.

    The NN handler count can sometimes stand to be increased, since all clients
    interacting with the NN are using Hadoop RPC. If it's set too low and you
    have many clients concurrently making calls to the NN, you'll see RPCs to
    the NN appear to have a high latency despite the NN not really being under
    much load. It defaults to 10, but increasing it to 50 or 100 or so is
    probably fine. If those threads are sitting idle on the NN then they'll
    take up a little NN memory, but not really harm anything other than that.

    --
    Aaron T. Myers
    Software Engineer, Cloudera


    On Tue, Dec 4, 2012 at 3:50 AM, Inder Pall wrote:

    Thx Muhammed saw this one however wanted more depth w.r.t this and
    dfs.namnode.handler.count as what are side-effects of a low setting...how
    do you know if the setting is good enough and stuff like that.

    - inder


    On Tue, Dec 4, 2012 at 5:02 PM, Muhamed Ghareeb <muhamed.ghareeb@gmail.com
    wrote:
    take a look for the below article

    http://blog.cloudera.com/blog/2009/03/configuration-parameters-what-can-you-just-ignore/



    On Tuesday, December 4, 2012 12:56:49 PM UTC+2, inder singh wrote:

    Can't find much information regarding this. How to arrive at a good
    value...does changing it effect the write throughput of a cluster.

    Any info/pointers would be highly appreciated.

    --
    - Inder
    "You are average of the 5 people you spend the most time with"

    --



    --
    - Inder
    "You are average of the 5 people you spend the most time with"

    --


    --
  • Dave Campbell at Oct 18, 2013 at 7:02 pm
    so i have a slightly related question, but may or not relate
    to dfs.datanode.handler.count.

    i have cluster that has recently been reduced from 8 datanodes to 3
    datanodes.
    this parameter on my namenode is set to 4.
    my namenode shows a dead datanode = localhost, and the 3 live datanodes
    that are defined in the slaves file.

    is that setting=4 causing the namenode to 'force' a dead datanode if there
    are only 3 real datanodes available?

    we are running CDH 20.0.

    <https://lh3.googleusercontent.com/-bHGs08yqMWM/UmGFoLyJ5zI/AAAAAAAAMs0/akTBSjC0Wzo/s1600/cap-+2013-10-18+at+15.00.59+.png>

    <https://lh5.googleusercontent.com/-lrhnfTyeD0I/UmGFsniVvlI/AAAAAAAAMs8/96pUTEs0P6A/s1600/cap-+2013-10-18+at+15.01.15+.png>


    On Tuesday, December 4, 2012 5:56:49 AM UTC-5, inder singh wrote:

    Can't find much information regarding this. How to arrive at a good
    value...does changing it effect the write throughput of a cluster.

    Any info/pointers would be highly appreciated.

    --
    - Inder
    "You are average of the 5 people you spend the most time with"
    --

    ---
    You received this message because you are subscribed to the Google Groups "CDH Users" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to cdh-user+unsubscribe@cloudera.org.
    For more options, visit https://groups.google.com/a/cloudera.org/groups/opt_out.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcdh-user @
categorieshadoop
postedDec 4, '12 at 10:56a
activeOct 18, '13 at 7:02p
posts5
users4
websitecloudera.com
irc#hadoop

People

Translate

site design / logo © 2022 Grokbase