FAQ
Hi,

Ive been trying to understand how the Balancer class works. I know
the basics about the threshold guidelines and the general motivations
of running the balancer.

I had a few questions about block migrations:

1. I noticed there is a Source and Target, but there is also a ProxySource.
I havent been able to grasp the conceptual difference between a source and a
proxy source

2. I'm also having trouble understanding the actual mechanics of a block
being relocated from
one datanode to the other. I understand that the communication with the
namenode occurs to change
metadata of the block being relocated accordingly, but have not understood
what does the actual copy
of the block itself.

I'd appreciate it if some developer can shed some light on this for me and
point me in the right direction
(like which classes/functions I should be looking at to understand this)

Thanks,
--
Amit Gupta

Search Discussions

  • Harsh J at Apr 24, 2011 at 2:43 pm
    Hello Amit,
    On Sun, Apr 24, 2011 at 5:00 AM, Amit Gupta wrote:
    1. I noticed there is a Source and Target, but there is also a ProxySource.
    I havent been able to grasp the conceptual difference between a source and a
    proxy source
    The 'proxy' source is a DN that has a replica of a given block. This
    is used in order to even out the load across the DNs in the network.
    It would not be a good idea to put too much tx load onto the breaching
    DN, when you can copy off a replica.
    2. I'm also having trouble understanding the actual mechanics of a block
    being relocated from
    one datanode to the other. I understand that the communication with the
    namenode occurs to change
    metadata of the block being relocated accordingly, but have not understood
    what does the actual copy
    of the block itself.
    The block movement is done from a DN to another DN directly when it
    receives a command from the NameNode to do so (part of the heartbeat
    mechanism).

    --
    Harsh J

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouphdfs-dev @
categorieshadoop
postedApr 23, '11 at 11:30p
activeApr 24, '11 at 2:43p
posts2
users2
websitehadoop.apache.org...
irc#hadoop

2 users in discussion

Amit Gupta: 1 post Harsh J: 1 post

People

Translate

site design / logo © 2022 Grokbase