Grokbase Groups Hive user August 2011
FAQ
if we have a very small table to be joined. we can use map side join and need the small table to be located on the map task. Is it possible to replicate the small table to ALL nodes when create the small table to cute the time to distribute the small table?

Search Discussions

  • Loren Siebert at Aug 12, 2011 at 2:50 am
    The Hive table is just a directory in HDFS, so you can recursively set the replication factor on it as you like. You can set it to the number of datanodes you have. If you have 100 nodes, then run this after you create your table:

    hadoop fs -setrep -R -w 100 /path/to/hive/warehouse/small_table_to_be_distributed
    On Aug 11, 2011, at 7:43 PM, Daniel,Wu wrote:

    if we have a very small table to be joined. we can use map side join and need the small table to be located on the map task. Is it possible to replicate the small table to ALL nodes when create the small table to cute the time to distribute the small table?

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupuser @
categorieshive, hadoop
postedAug 12, '11 at 2:43a
activeAug 12, '11 at 2:50a
posts2
users2
websitehive.apache.org

2 users in discussion

Loren Siebert: 1 post Daniel,Wu: 1 post

People

Translate

site design / logo © 2021 Grokbase