FAQ
hi,

In my custom partitioner (which may assign a key to more than one partitions), I
want to sometimes send a key to more than one reducers. But the default
getPartition method provided by hadoop allows to send only one value for a key.
how can I get around this?

Thanks

Search Discussions

  • Gang Luo at Jul 26, 2010 at 7:19 pm
    you have only to duplicate the record, any by whatever method return a different
    value from getPartition for each of them.


    -Gang



    ----- 原始邮件 ----
    发件人: abc xyz <fabc_xyz111@yahoo.com>
    收件人: common-user@hadoop.apache.org
    发送日期: 2010/7/26 (周一) 10:08:20 上午
    主 题: sending a key-value pair to more than one reducers

    hi,

    In my custom partitioner (which may assign a key to more than one partitions), I

    want to sometimes send a key to more than one reducers. But the default
    getPartition method provided by hadoop allows to send only one value for a key.
    how can I get around this?

    Thanks
  • Gang Luo at Jul 26, 2010 at 7:37 pm
    Hi all,
    assume some of my files are empty (size is 0) and I name them as the input to my
    MR job, will a map task be launched on each of them?

    Thanks,
    -Gang
  • Ted Yu at Jul 26, 2010 at 7:48 pm
    No.
    There is no InputSplit generated from them.
    On Mon, Jul 26, 2010 at 12:37 PM, Gang Luo wrote:

    Hi all,
    assume some of my files are empty (size is 0) and I name them as the input
    to my
    MR job, will a map task be launched on each of them?

    Thanks,
    -Gang



Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-user @
categorieshadoop
postedJul 26, '10 at 2:09p
activeJul 26, '10 at 7:48p
posts4
users3
websitehadoop.apache.org...
irc#hadoop

3 users in discussion

Gang Luo: 2 posts Ted Yu: 1 post Abc xyz: 1 post

People

Translate

site design / logo © 2022 Grokbase