Hi,



I have a couple of questions:



1. What is the best way to create a composed MapReduce job in the 20.2
API? Can you use JobControl, which is still located in the mapred namespace,
or is it better to avoid mixing API's?

2. Has anyone ever worked with composed MapReduce jobs on Amazon
Elastic MapReduce (20.2 API)?



Regards,

Joachim

Search Discussions

  • Harsh J at Feb 16, 2011 at 10:34 am
    Hey,

    On Wed, Feb 16, 2011 at 2:02 PM, Joachim Van den Bogaert
    wrote:
    Hi,



    I have a couple of questions:



    1.       What is the best way to create a composed MapReduce job in the 20.2
    API? Can you use JobControl, which is still located in the mapred namespace,
    or is it better to avoid mixing API’s?
    IMO, it is never a good idea to mix old/new APIs. In 0.20.x, you don't
    have a mapreduce.lib.jobcontrol to do this; you would need 0.21.

    You may want to check Yahoo!'s Oozie or LinkedIn's Azkaban for
    building workflow constructs out of a set of MapReduce jobs.

    P.s. The 'deprecated' API in 0.20.2 has been un-deprecated in 0.20.3;
    you can consider using it as it is still supported as a stable API in
    0.21 (and perhaps 0.22 too).

    --
    Harsh J
    www.harshj.com

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupmapreduce-user @
categorieshadoop
postedFeb 16, '11 at 8:32a
activeFeb 16, '11 at 10:34a
posts2
users2
websitehadoop.apache.org...
irc#hadoop

People

Translate

site design / logo © 2023 Grokbase