FAQ
Hi all,

Can anyone point me to a simple YARN-based MR example - one that uses a
client to submit jobs to the platform. The example here
http://hadoop.apache.org/common/docs/r0.23.0/hadoop-yarn/hadoop-yarn-site/WritingYarnApplications.html is
awfully outdated, with so many bugs and omissions.

Thanks,
Thinus

Search Discussions

  • Philip Zeyliger at Jul 24, 2012 at 3:55 pm
    Hi Thinus,

    If you're using "MR2", the client API is actually the same. Be sure you
    set HADOOP_MAPRED_HOME as described in
    https://ccp.cloudera.com/display/CDH4DOC/Deploying+MapReduce+v2+(YARN)+on+a+Cluster#DeployingMapReducev2%28YARN%29onaCluster-Step13
    .

    Cheers,

    -- Philip

    On Tue, Jul 24, 2012 at 2:57 AM, Thinus Prinsloo
    wrote:
    Hi all,

    Can anyone point me to a simple YARN-based MR example - one that uses a
    client to submit jobs to the platform. The example here
    http://hadoop.apache.org/common/docs/r0.23.0/hadoop-yarn/hadoop-yarn-site/WritingYarnApplications.html is
    awfully outdated, with so many bugs and omissions.

    Thanks,
    Thinus
  • Thinus Prinsloo at Jul 25, 2012 at 4:53 pm
    Thanks Philip,

    I was more referring to the Client/ApplicationMaster sort of
    implementation. I came right, in a way, by looking at the Distributed
    Shell example in the source. If only I can get it to work ...

    Thanks anyhow. For those who are interested, check out
    "hadoop-mapreduce-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell".
    On Tuesday, 24 July 2012 17:55:08 UTC+2, Philip Zeyliger wrote:

    Hi Thinus,

    If you're using "MR2", the client API is actually the same. Be sure you
    set HADOOP_MAPRED_HOME as described in
    https://ccp.cloudera.com/display/CDH4DOC/Deploying+MapReduce+v2+(YARN)+on+a+Cluster#DeployingMapReducev2%28YARN%29onaCluster-Step13
    .

    Cheers,

    -- Philip
  • Philip Zeyliger at Jul 25, 2012 at 4:28 pm
    Ah. You might check out Josh Wills' work on Kitten ("plays with yarn") at
    http://www.cloudera.com/blog/2012/06/hadoop-beyond-mapreduce-introducing-kitten/
    .

    -- Philip

    On Wed, Jul 25, 2012 at 8:13 AM, Thinus Prinsloo
    wrote:
    Thanks Philip,

    I was more referring to the Client/ApplicationMaster sort of
    implementation. I came right, in a way, by looking at the Distributed
    Shell example in the source. If only I can get it to work ...

    Thanks anyhow. For those who are interested, check out
    "hadoop-mapreduce-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell".
    On Tuesday, 24 July 2012 17:55:08 UTC+2, Philip Zeyliger wrote:

    Hi Thinus,

    If you're using "MR2", the client API is actually the same. Be sure you
    set HADOOP_MAPRED_HOME as described in https://ccp.cloudera.com/**
    display/CDH4DOC/Deploying+**MapReduce+v2+(YARN)+on+a+**
    Cluster#DeployingMapReducev2%**28YARN%29onaCluster-Step13<https://ccp.cloudera.com/display/CDH4DOC/Deploying+MapReduce+v2+(YARN)+on+a+Cluster#DeployingMapReducev2%28YARN%29onaCluster-Step13>
    .

    Cheers,

    -- Philip
  • Thinus Prinsloo at Jul 26, 2012 at 9:58 am
    Thanks, looks interesting. Struggling to just get the built-in stuff going
    (posted a new query about that).

    T.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupscm-users @
categorieshadoop
postedJul 24, '12 at 9:57a
activeJul 26, '12 at 9:58a
posts5
users2
websitecloudera.com
irc#hadoop

People

Translate

site design / logo © 2022 Grokbase