Grokbase Groups Pig user March 2010
FAQ
Hi all,

Is there any way to pass -Dmapred.map.tasks to pig? I notice that my
multi-core nodes are under-utilized with pig job, only processor is busy all
the time.
I notice mapred.map.tasks is set to # of nodes from Hadoop job console. I
try to change config under $HADOOP/conf, does not seem to work. Overwritten
by Pig??

Please advise. Thanks

Search Discussions

  • Daniel Dai at Mar 22, 2010 at 4:16 pm
    If you are using pig 0.6 and under, use -Dpig.overrideBlockSize. If you are
    using pig trunk (0.7), use -Dmapred.min.split.size

    Daniel

    --------------------------------------------------
    From: "Qiming He" <qiming.he@openresearchinc.com>
    Sent: Saturday, March 20, 2010 9:48 PM
    To: <pig-user@hadoop.apache.org>
    Subject: pass mapred.map.tasks to pig
    Hi all,

    Is there any way to pass -Dmapred.map.tasks to pig? I notice that my
    multi-core nodes are under-utilized with pig job, only processor is busy
    all
    the time.
    I notice mapred.map.tasks is set to # of nodes from Hadoop job console. I
    try to change config under $HADOOP/conf, does not seem to work.
    Overwritten
    by Pig??

    Please advise. Thanks

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupuser @
categoriespig, hadoop
postedMar 21, '10 at 4:24p
activeMar 22, '10 at 4:16p
posts2
users2
websitepig.apache.org

2 users in discussion

Daniel Dai: 1 post Qiming He: 1 post

People

Translate

site design / logo © 2021 Grokbase