Thank you a lot for the replies.

To me it is clear when data locality gets broken though (and it is not only
the failure of the RS, there are other cases). I was hoping more for
suggestions around this particular use-case: assuming that nodes/RSs are
stable, how to make sure to achieve the data locality when doing bulk
import (writing HFiles directly from MR job). Running major compaction
helps here (as new files are created instead of old ones *on the DataNode
local to RS where region is being compacted), but I'd really want to not do
it. This is quite resource intensive and thus expensive process...

I was hoping also guys from HDFS/MapReduce teams would comment on my latter

I heard that there is some work in HBase community to allow "asking" HDFS
to replicate blocks of the files together (so that there are full replicas
on other nodes, which helps as Lars noted) too. I also heard from a HDFS
guy that there are ideas around better replication logic.

Little offtop:
Also is it correct to say that if i set smaller data block size data
locality gets worse, and if data block size gets bigger data
*Theoretically* if your region data stored in one HFile (say one flush
occurred or major compaction caused that, given that there's one CF) and
this HFile is smaller than the configured block size on HDFS, then we can
say that 3 (or whatever is replication) replicas of this file (and hence
of this region) are "full" replicas, which makes it easier to preserve data
locality if RS fails down (or when anything else cause re-assigning the
region). But since Region size is usually much bigger (usually 10-20 times
bigger at least), this fact doesn't buy you something.

Alex Baranau
Sematext :: http://blog.sematext.com/ :: Hadoop - HBase - ElasticSearch -
On Wed, Jul 18, 2012 at 9:43 PM, Ben Kim wrote:

I added some Q&A's went with Lars. Hope this is somewhat related to your
data locality questions.
On Jun 15, 2012, at 6:56 AM, Ben Kim wrote:


I've been posting questions in the mailing-list quiet often lately,
here goes another one about data locality
I read the excellent blog post about data locality that Lars George
I understand data locality in hbase as locating a region in a
where most of its data blocks reside.
The opposite is happening, i.e. the region server process triggers
for all
data it writes to be located on the same physical machine.
So that way fast data access is guranteed when running a MR because
map/reduce task is run for each region in the tasktracker where the region
co-locates. Correct.
But what if the data blocks of the region are evenly spread over
This will not happen, unless the original server fails. Then the
region is
moved to another that now needs to do a lot of remote reads over the
network. This is way there is work being done to allow for custom
policies in HDFS. That way you can store the entire region and all
as complete units on three data nodes. In case of a failure you can
move the region to one of the two copies. This is not available yet
but it is being worked on (so I heard).
Does a MR task has to remotely access the data blocks from other
For the above failure case, it would be the region server accessing
remote data, yes.
How good is hbase locating datablocks where a region resides?
That is again the wrong way around. HBase has no clue as to where
reside, nor does it know that the file system in fact uses separate
HBase stores files, HDFS does the block magic underneath the hood,
transparent to HBase.
Also is it correct to say that if i set smaller data block size
locality gets worse, and if data block size gets bigger data
This is not applicable here, I am assuming this stems from the above
confusion about which system is handling the blocks, HBase or HDFS.


On Thu, Jul 19, 2012 at 6:39 AM, Cristofer Weber <
cristofer.weber@neogrid.com> wrote:
Hi Alex,

I ran one of our bulk import jobs with partial payload, without
with major compaction, and you are right: Some hdfs blocks are in a
different datanode.

-----Mensagem original-----
De: Alex Baranau
Enviada em: quarta-feira, 18 de julho de 2012 12:46
Para: hbase-user@hadoop.apache.org; mapreduce-user@hadoop.apache.org;
Assunto: Bulk Import & Data Locality


As far as I understand Bulk Import functionality will not take into
account the Data Locality question. MR job will create number of reducer
tasks same as regions to write into, but it will not "advice" on which
nodes to run these tasks. In that case Reducer task which writes HFiles of
some region may not be physically located at the same node as RS that
serves that region. The way HDFS writes data, there will be (likely) one
full replica of bolcks of HFiles of this Region written on the node where
Reducer task was run and other replicas (if replication >1) will be
distributed randomly over the cluster. Thus, RS while serving data of that
region will (most
likely) not look at local data (data will be transferred from other
datanodes). I.e. data locality will be broken.

Is this correct?

If yes, I guess, if we could tell MR framework where (which nodes) to
launch certain Reducer tasks, this would help us. I believe this is not
possible with MR1, please correct me if I'm wrong. Perhaps, this is this
possible with MR2?

I assume there's no way to provide a "hint" to a NameNode where to place
blocks of a new File too, right?

Thank you,
Alex Baranau
Sematext :: http://blog.sematext.com/ :: Hadoop - HBase - ElasticSearch -


*Benjamin Kim*
*benkimkimben at gmail*

Alex Baranau
Sematext :: http://blog.sematext.com/ :: Hadoop - HBase - ElasticSearch -

Alex Baranau
Sematext :: http://blog.sematext.com/ :: Hadoop - HBase - ElasticSearch -

Search Discussions

Discussion Posts


Related Discussions

Discussion Navigation
viewthread | post
posts ‹ prev | 2 of 2 | next ›
Discussion Overview
groupmapreduce-user @
postedJul 18, '12 at 3:46p
activeJul 19, '12 at 2:55a

1 user in discussion

Alex Baranau: 2 posts



site design / logo © 2021 Grokbase