FAQ
Hello,

I have a problem getting the map input file name. Here is what I tried:

public class Map extends Mapper<Object, Text, LongWritable, Text> {

public void map(Object key, Text value, Context context)
throws IOException, InterruptedException {
Configuration conf = context.getConfiguration();
System.out.println(conf.get("map.input.file"));
System.out.println(conf.get("map.input.filename"));
System.out.println(conf.get("mapred.input.file"));
System.out.println(conf.get("mapred.input.filename"));
}

protected void setup(Context context) throws IOException,
InterruptedException {
Configuration conf = context.getConfiguration();
System.out.println(conf.get("map.input.file"));
System.out.println(conf.get("map.input.filename"));
System.out.println(conf.get("mapred.input.file"));
System.out.println(conf.get("mapred.input.filename"));
}
}

All the get is "null". What am I doing wrong?

Thanks,
Rares

Search Discussions

  • Randysch at Jun 2, 2009 at 4:51 pm
    Your Map class needs to have a configure method that can access the JobConf. Like this:

    public void configure(JobConf conf)
    {
    System.out.println(conf.get("map.input.file");
    }


    randy

    ----- Original Message -----
    From: "Rares Vernica" <rvernica@gmail.com>
    To: core-user@hadoop.apache.org
    Sent: Tuesday, June 2, 2009 12:45:02 PM GMT -05:00 US/Canada Eastern
    Subject: problem getting map input filename

    Hello,

    I have a problem getting the map input file name. Here is what I tried:

    public class Map extends Mapper<Object, Text, LongWritable, Text> {

    public void map(Object key, Text value, Context context)
    throws IOException, InterruptedException {
    Configuration conf = context.getConfiguration();
    System.out.println(conf.get("map.input.file"));
    System.out.println(conf.get("map.input.filename"));
    System.out.println(conf.get("mapred.input.file"));
    System.out.println(conf.get("mapred.input.filename"));
    }

    protected void setup(Context context) throws IOException,
    InterruptedException {
    Configuration conf = context.getConfiguration();
    System.out.println(conf.get("map.input.file"));
    System.out.println(conf.get("map.input.filename"));
    System.out.println(conf.get("mapred.input.file"));
    System.out.println(conf.get("mapred.input.filename"));
    }
    }

    All the get is "null". What am I doing wrong?

    Thanks,
    Rares
  • Rares Vernica at Jun 2, 2009 at 5:52 pm

    On 6/2/09, randysch@comcast.net wrote:
    Your Map class needs to have a configure method that can access the JobConf.
    Like this:
    In Hadoop 0.20 the new Mapper class does no longer have "configure"
    and JobConf has been replaced with Job. In the Mapper methods, you now
    get a Context object.
    public void configure(JobConf conf)
    {
    System.out.println(conf.get("map.input.file");
    }
  • Jason hadoop at Jun 3, 2009 at 3:13 am
    you can always dump the entire property space and work it out that way.
    I haven't used the 0.20 api's yet so I can't speak to them
    On Tue, Jun 2, 2009 at 10:52 AM, Rares Vernica wrote:
    On 6/2/09, randysch@comcast.net wrote:

    Your Map class needs to have a configure method that can access the JobConf.
    Like this:
    In Hadoop 0.20 the new Mapper class does no longer have "configure"
    and JobConf has been replaced with Job. In the Mapper methods, you now
    get a Context object.
    public void configure(JobConf conf)
    {
    System.out.println(conf.get("map.input.file");
    }


    --
    Alpha Chapters of my book on Hadoop are available
    http://www.apress.com/book/view/9781430219422
    www.prohadoopbook.com a community for Hadoop Professionals
  • Rares Vernica at Jun 3, 2009 at 4:35 pm

    On 6/2/09, jason hadoop wrote:
    you can always dump the entire property space and work it out that way.
    I dumped the property space and I could only find "mapred.input.dir".
    There was no "mapred.input.file".

    --
    Rares
  • He Yongqiang at Jun 3, 2009 at 4:55 pm
    take a look at HADOOP-5368, :)
    On 09-6-4 上午12:27, "Rares Vernica" wrote:
    On 6/2/09, jason hadoop wrote:

    you can always dump the entire property space and work it out that way.
    I dumped the property space and I could only find "mapred.input.dir".
    There was no "mapred.input.file".

    --
    Rares
  • Rares Vernica at Jun 3, 2009 at 6:12 pm

    On 6/3/09, He Yongqiang wrote:
    take a look at HADOOP-5368, :)
    There you set the "map.input.file", I think it should already be set by Hadoop.
  • Sharad Agarwal at Jun 3, 2009 at 7:32 am
    conf.get("map.input.file") should work. If not, then it is a bug in new mapreduce api in 0.20

    - Sharad
  • Rares Vernica at Jun 4, 2009 at 3:06 pm

    On 6/2/09, Rares Vernica wrote:
    I have a problem getting the map input file name. Here is what I tried:

    public class Map extends Mapper<Object, Text, LongWritable, Text> {

    public void map(Object key, Text value, Context context)
    throws IOException, InterruptedException {
    Configuration conf = context.getConfiguration();
    System.out.println(conf.get("map.input.file"));
    System.out.println(conf.get("map.input.filename"));
    System.out.println(conf.get("mapred.input.file"));
    System.out.println(conf.get("mapred.input.filename"));
    }

    protected void setup(Context context) throws IOException,
    InterruptedException {
    Configuration conf = context.getConfiguration();
    System.out.println(conf.get("map.input.file"));
    System.out.println(conf.get("map.input.filename"));
    System.out.println(conf.get("mapred.input.file"));
    System.out.println(conf.get("mapred.input.filename"));
    }
    }

    All the get is "null". What am I doing wrong?
    I created https://issues.apache.org/jira/browse/HADOOP-5973

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-user @
categorieshadoop
postedJun 2, '09 at 4:45p
activeJun 4, '09 at 3:06p
posts9
users5
websitehadoop.apache.org...
irc#hadoop

People

Translate

site design / logo © 2022 Grokbase