FAQ
Hi,

I need to be able to retrieve the current job id from within map or
reduce tasks. I looked here and there, but I wasn't able to find any way
to do this in the API ...

Basically, I want to be able to do something like this:

class XXX implements Mapper {

public void configure(Configuration conf) {

String jobId = ... ?
String taskId = ... ?
}
}

And similarly:


class XXX implements Reducer {

public void configure(Configuration conf) {

String jobId = ... ?
String taskId = ... ?
}
}

Is this possible?

--
Best regards,
Andrzej Bialecki <><
___. ___ ___ ___ _ _ __________________________________
[__ || __|__/|__||\/| Information Retrieval, Semantic Web
___|||__|| \| || | Embedded Unix, System Integration
http://www.sigram.com Contact: info at sigram dot com

Search Discussions

  • Owen O'Malley at Sep 13, 2006 at 4:07 pm

    On Sep 13, 2006, at 7:07 AM, Andrzej Bialecki wrote:

    Hi,

    I need to be able to retrieve the current job id from within map or
    reduce tasks. I looked here and there, but I wasn't able to find
    any way to do this in the API ...
    Yes, look under "Localized Properties in the JobConf" on the wiki:

    http://wiki.apache.org/lucene-hadoop/TaskExecutionEnvironment

    In particular, conf.get("mapred.job.id") and conf.get
    ("mapred.task.id") will do the job.

    -- Owen
  • Doug Cutting at Sep 13, 2006 at 5:19 pm

    Owen O'Malley wrote:
    In particular, conf.get("mapred.job.id") and conf.get("mapred.task.id")
    will do the job.
    Should we add JobConf methods for these?

    Doug
  • Owen O'Malley at Sep 13, 2006 at 5:37 pm

    On Sep 13, 2006, at 10:17 AM, Doug Cutting wrote:

    Owen O'Malley wrote:
    In particular, conf.get("mapred.job.id") and conf.get
    ("mapred.task.id") will do the job.
    Should we add JobConf methods for these?
    We should probably have a TaskConf that does:

    public class TaskConf extends JobConf {
    public String getJobId() {...}
    public String getTaskId() {...}
    }

    since they don't really make much sense except for the object that is
    passed down to the tasks.
  • Doug Cutting at Sep 13, 2006 at 5:48 pm

    Owen O'Malley wrote:
    We should probably have a TaskConf that does:

    public class TaskConf extends JobConf {
    public String getJobId() {...}
    public String getTaskId() {...}
    }

    since they don't really make much sense except for the object that is
    passed down to the tasks.
    Static methods might be simpler. These could be on TaskTracker, or a
    new class just for this purpose.

    Doug
  • Andrzej Bialecki at Sep 13, 2006 at 8:26 pm

    Owen O'Malley wrote:
    On Sep 13, 2006, at 7:07 AM, Andrzej Bialecki wrote:

    Hi,

    I need to be able to retrieve the current job id from within map or
    reduce tasks. I looked here and there, but I wasn't able to find any
    way to do this in the API ...
    Yes, look under "Localized Properties in the JobConf" on the wiki:

    http://wiki.apache.org/lucene-hadoop/TaskExecutionEnvironment

    In particular, conf.get("mapred.job.id") and
    conf.get("mapred.task.id") will do the job.
    Thanks Owen, I should have checked this page ...

    --
    Best regards,
    Andrzej Bialecki <><
    ___. ___ ___ ___ _ _ __________________________________
    [__ || __|__/|__||\/| Information Retrieval, Semantic Web
    ___|||__|| \| || | Embedded Unix, System Integration
    http://www.sigram.com Contact: info at sigram dot com

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-dev @
categorieshadoop
postedSep 13, '06 at 2:08p
activeSep 13, '06 at 8:26p
posts6
users3
websitehadoop.apache.org...
irc#hadoop

People

Translate

site design / logo © 2022 Grokbase