FAQ
Hi,

according to the API-Dokumentation of 0.20.1 JobConf is deprecated and
we should use Configuration instead. However all examples on the webpage
still referece JobConf.

Is there a good example for replacing JobConf by Configuration?

Regards, Oliver

- --
Oliver B. Fischer, Schönhauser Allee 64, 10437 Berlin
Tel. +49 30 44793251, Mobil: +49 178 7903538
Mail: o.b.fischer@swe-blog.net Blog: http://www.swe-blog.net

Search Discussions

  • Mark Vigeant at Oct 21, 2009 at 2:15 pm
    Hi Oliver,

    I ran into the same problem a few weeks ago. What you want to do now is use the classes Job and Configuration. Here's basically what you need to do:


    Configuration conf = new Configuration(); //creates the configuration object
    conf.set(TableOutputFormat.OUTPUT_TABLE, name); //set the output format: I'm using this for HBase so it's an HTable.
    Job job = new Job(conf, NAME);
    job.setJarByClass(TableUploader.class);
    job.setInputFormatClass(FileInputFormat.class);
    job.setMapperClass(MapExample.class);
    job.setNumReduceTasks(0); //alternately, you could set a Reducer class. Check the API
    FileInputFormat.setInputPaths(job, new Path(input));
    job.setOutputFormatClass(TableOutputFormat.class);
    job.waitForCompletion(true); //submits the job, waits for it to be completed.

    -----Original Message-----
    From: Oliver B. Fischer
    Sent: Wednesday, October 21, 2009 8:49 AM
    To: common-user@hadoop.apache.org
    Subject: Using Configuration instead of JobConf

    -----BEGIN PGP SIGNED MESSAGE-----
    Hash: SHA1

    Hi,

    according to the API-Dokumentation of 0.20.1 JobConf is deprecated and
    we should use Configuration instead. However all examples on the webpage
    still referece JobConf.

    Is there a good example for replacing JobConf by Configuration?

    Regards, Oliver

    - --
    Oliver B. Fischer, Schönhauser Allee 64, 10437 Berlin
    Tel. +49 30 44793251, Mobil: +49 178 7903538
    Mail: o.b.fischer@swe-blog.net Blog: http://www.swe-blog.net
  • Ryan Smith at Oct 21, 2009 at 8:02 pm
    It would be great if someone could update the src code here:

    http://hadoop.apache.org/common/docs/current/mapred_tutorial.html#Source+Code
    On Wed, Oct 21, 2009 at 10:15 AM, Mark Vigeant wrote:

    Hi Oliver,

    I ran into the same problem a few weeks ago. What you want to do now is use
    the classes Job and Configuration. Here's basically what you need to do:


    Configuration conf = new Configuration(); //creates the configuration
    object
    conf.set(TableOutputFormat.OUTPUT_TABLE, name); //set the output format:
    I'm using this for HBase so it's an HTable.
    Job job = new Job(conf, NAME);
    job.setJarByClass(TableUploader.class);
    job.setInputFormatClass(FileInputFormat.class);
    job.setMapperClass(MapExample.class);
    job.setNumReduceTasks(0); //alternately, you could set a Reducer class.
    Check the API
    FileInputFormat.setInputPaths(job, new Path(input));
    job.setOutputFormatClass(TableOutputFormat.class);
    job.waitForCompletion(true); //submits the job, waits for it to be
    completed.

    -----Original Message-----
    From: Oliver B. Fischer
    Sent: Wednesday, October 21, 2009 8:49 AM
    To: common-user@hadoop.apache.org
    Subject: Using Configuration instead of JobConf

    -----BEGIN PGP SIGNED MESSAGE-----
    Hash: SHA1

    Hi,

    according to the API-Dokumentation of 0.20.1 JobConf is deprecated and
    we should use Configuration instead. However all examples on the webpage
    still referece JobConf.

    Is there a good example for replacing JobConf by Configuration?

    Regards, Oliver

    - --
    Oliver B. Fischer, Schönhauser Allee 64, 10437 Berlin
    Tel. +49 30 44793251, Mobil: +49 178 7903538
    Mail: o.b.fischer@swe-blog.net Blog: http://www.swe-blog.net
    -----BEGIN PGP SIGNATURE-----
    Version: GnuPG v1.4.9 (MingW32)
    Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

    iQEcBAEBAgAGBQJK3wNFAAoJELeiwuwqd1DGf4YH/3P83TQh+7L0GiKT4rdIe1HC
    LdZ1weVcqnBKfFZYCsL64PfwPGhkl0hoTU/sU29qGGrnAN8QKbF/8m2P32RlysCV
    ONT2nUlugHfEb8iQzF8Xx3uuCzU3AXYHK0eqme+thzfscxZKiGKzLTYb26qFnufG
    AtTIEKMkv5SRLIn5YRFM/49dcSp2WIRCIEcc+0eofsxFvI9X2a8Sg8AMG5JzNqYT
    SHMGSr3boBc4gFCNTCsRmJQIusPVWyJZICVss0q3491xdMVvGfAPTsbb6BLINFkL
    j5BrbC9OaNnmX3X3kR2/orABwMj9y5Xd1YjXepeWG3QD3WftPVyoCIOE1v8m2s0=
    =OFkK
    -----END PGP SIGNATURE-----
  • Ryan Smith at Oct 21, 2009 at 8:12 pm
    Also, FileInputFormat is deprecated in hadoop 0.20.1.
    On Wed, Oct 21, 2009 at 10:15 AM, Mark Vigeant wrote:

    Hi Oliver,

    I ran into the same problem a few weeks ago. What you want to do now is use
    the classes Job and Configuration. Here's basically what you need to do:


    Configuration conf = new Configuration(); //creates the configuration
    object
    conf.set(TableOutputFormat.OUTPUT_TABLE, name); //set the output format:
    I'm using this for HBase so it's an HTable.
    Job job = new Job(conf, NAME);
    job.setJarByClass(TableUploader.class);
    job.setInputFormatClass(FileInputFormat.class);
    job.setMapperClass(MapExample.class);
    job.setNumReduceTasks(0); //alternately, you could set a Reducer class.
    Check the API
    FileInputFormat.setInputPaths(job, new Path(input));
    job.setOutputFormatClass(TableOutputFormat.class);
    job.waitForCompletion(true); //submits the job, waits for it to be
    completed.

    -----Original Message-----
    From: Oliver B. Fischer
    Sent: Wednesday, October 21, 2009 8:49 AM
    To: common-user@hadoop.apache.org
    Subject: Using Configuration instead of JobConf

    -----BEGIN PGP SIGNED MESSAGE-----
    Hash: SHA1

    Hi,

    according to the API-Dokumentation of 0.20.1 JobConf is deprecated and
    we should use Configuration instead. However all examples on the webpage
    still referece JobConf.

    Is there a good example for replacing JobConf by Configuration?

    Regards, Oliver

    - --
    Oliver B. Fischer, Schönhauser Allee 64, 10437 Berlin
    Tel. +49 30 44793251, Mobil: +49 178 7903538
    Mail: o.b.fischer@swe-blog.net Blog: http://www.swe-blog.net
    -----BEGIN PGP SIGNATURE-----
    Version: GnuPG v1.4.9 (MingW32)
    Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

    iQEcBAEBAgAGBQJK3wNFAAoJELeiwuwqd1DGf4YH/3P83TQh+7L0GiKT4rdIe1HC
    LdZ1weVcqnBKfFZYCsL64PfwPGhkl0hoTU/sU29qGGrnAN8QKbF/8m2P32RlysCV
    ONT2nUlugHfEb8iQzF8Xx3uuCzU3AXYHK0eqme+thzfscxZKiGKzLTYb26qFnufG
    AtTIEKMkv5SRLIn5YRFM/49dcSp2WIRCIEcc+0eofsxFvI9X2a8Sg8AMG5JzNqYT
    SHMGSr3boBc4gFCNTCsRmJQIusPVWyJZICVss0q3491xdMVvGfAPTsbb6BLINFkL
    j5BrbC9OaNnmX3X3kR2/orABwMj9y5Xd1YjXepeWG3QD3WftPVyoCIOE1v8m2s0=
    =OFkK
    -----END PGP SIGNATURE-----
  • Oliver B. Fischer at Oct 27, 2009 at 1:43 pm

    Oliver B. Fischer schrieb:
    Hi,

    according to the API-Dokumentation of 0.20.1 JobConf is deprecated and
    we should use Configuration instead. However all examples on the webpage
    still referece JobConf.

    Is there a good example for replacing JobConf by Configuration?

    Regards, Oliver
    Hello, I created HADOOP-6336 for this.


    - --
    Oliver B. Fischer, Schönhauser Allee 64, 10437 Berlin
    Tel. +49 30 44793251, Mobil: +49 178 7903538
    Mail: o.b.fischer@swe-blog.net Blog: http://www.swe-blog.net
  • Tim robertson at Oct 27, 2009 at 1:47 pm
    The org.apache.hadoop.examples.SecondarySort in 0.20.1 is an example
    using the org.apache.hadoop.conf.Configuration.

    Cheers,
    Tim

    On Tue, Oct 27, 2009 at 2:42 PM, Oliver B. Fischer
    wrote:
    -----BEGIN PGP SIGNED MESSAGE-----
    Hash: SHA1

    Oliver B. Fischer schrieb:
    Hi,

    according to the API-Dokumentation of 0.20.1 JobConf is deprecated and
    we should use Configuration instead. However all examples on the webpage
    still referece JobConf.

    Is there a good example for replacing JobConf by Configuration?

    Regards, Oliver
    Hello, I created HADOOP-6336 for this.


    - --
    Oliver B. Fischer, Schönhauser Allee 64, 10437 Berlin
    Tel. +49 30 44793251, Mobil: +49 178 7903538
    Mail: o.b.fischer@swe-blog.net Blog: http://www.swe-blog.net
    -----BEGIN PGP SIGNATURE-----
    Version: GnuPG v1.4.9 (MingW32)
    Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

    iQEcBAEBAgAGBQJK5vjUAAoJELeiwuwqd1DG3m8H/RmFTwvLvHEftRDCfA4q1d5M
    +M4XST8Yw3grBLbrJDCorZXirzvG0PskgWfGs4o7Dvhq2zPcF4uubcuqi8ej/Wxf
    Lo9I+LQdkwuXNdwWMASMODpyPEBtwiQOAT/QUpHD9b7mpzzDEealRJz+GP96yLaH
    PaWgBj5F3d7KuI9AsdsXPBr3WMc+7CAukL/G8Wz4UN4k8Z8hc7/yJq5ISFUB+NrR
    z/NdSVJhSQfwIB44826bmeNgEAIKbZVGFZs4pzuicXG0Q6bs77GX83pcne4r1Mmv
    RWYIzKbMUUDqF7TT1h4BsBE54crf5TabJWwmFqHiH9NaR8fOBYcM8VBlT/XbdyM=
    =MT4v
    -----END PGP SIGNATURE-----
  • Oliver B. Fischer at Oct 29, 2009 at 4:26 am
    Hello Tim,

    thank you. I found the examples and even the WordCount has been updated.
    Only the documentation is out of date.

    Bye,

    Oliver

    tim robertson schrieb:
    The org.apache.hadoop.examples.SecondarySort in 0.20.1 is an example
    using the org.apache.hadoop.conf.Configuration.
    - --
    Oliver B. Fischer, Schönhauser Allee 64, 10437 Berlin
    Tel. +49 30 44793251, Mobil: +49 178 7903538
    Mail: o.b.fischer@swe-blog.net Blog: http://www.swe-blog.net

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-user @
categorieshadoop
postedOct 21, '09 at 12:49p
activeOct 29, '09 at 4:26a
posts7
users4
websitehadoop.apache.org...
irc#hadoop

People

Translate

site design / logo © 2022 Grokbase