FAQ
Good evening,



I would like to ask you a question regarding the use of TotalOrderPartitioner.
I am working on my diploma thesis, and I need to use the
TotalOrderPartitioner (with the InputSampler of course), under Hadoop
0.20.2

In
order to use it, I need to apply the patch
(https://issues.apache.org/jira/browse/MAPREDUCE-366), but
it fails for some reason.

If I am correct, the patch modifies the TotalOrderPartitioner & InputSampler classes in the org.apache.hadoop.mapred.lib package, in order to deprecate them and then it specifies 2 new classes to be used: TotalOrderPartitioner & InputSampler in org.apache.hadoop.mapreduce.lib.partitioner, using the new API.


I would like to ask, if someone has successfully applied the patch. Could he send me the new classes
(TotalOrderPartitioner and InputSampler) from their hadoop installation, after the patch is applied? (it affects the 2 classes
both in org.apache.hadoop.mapred.lib and
org.apache.hadoop.mapreduce.lib.partitioner packages). Or at least you
could suggest another solution?


I hope this will not consume your time. I apologize for the inconvenience, but I
need these two classes in order to finish my diploma thesis, and I don't know from who I should ask for help.


Thank you very much in advance,
Sofia Georgiakaki
undergraduate student
department of Electronic & Computer Engineering
Technical University of Crete, Greece

Search Discussions

  • Harsh J at Aug 3, 2011 at 1:10 pm
    Sofia,

    I'd recommend using the old (actually, stable) API for development
    right now, when using 0.20.2. Do not be confused by the deprecation
    marks since it has been un-deprecated for later releases. Using the
    stable API should rid you of the trouble of patching the whole thing
    up.

    I.e., use JobConf+JobClient+'mapred'-package to build and run jobs
    instead of the 'Job' class and 'mapreduce' package.

    On Wed, Aug 3, 2011 at 6:05 PM, Sofia Georgiakaki
    wrote:
    Good evening,



    I would like to ask you a question regarding the use of TotalOrderPartitioner.
    I am working on my diploma thesis, and I need to use the
    TotalOrderPartitioner (with the InputSampler of course), under Hadoop
    0.20.2

    In
    order to use it, I need to apply the patch
    (https://issues.apache.org/jira/browse/MAPREDUCE-366), but
    it fails for some reason.

    If I am correct, the patch modifies the TotalOrderPartitioner & InputSampler classes in  the org.apache.hadoop.mapred.lib package, in order to deprecate them and then it specifies 2 new classes to be used: TotalOrderPartitioner & InputSampler in org.apache.hadoop.mapreduce.lib.partitioner, using the new API.


    I would like to ask, if someone has successfully applied the patch. Could he send me the new classes
    (TotalOrderPartitioner and InputSampler) from their hadoop installation, after the patch is applied? (it affects the 2 classes
    both in org.apache.hadoop.mapred.lib and
    org.apache.hadoop.mapreduce.lib.partitioner packages). Or at least you
    could suggest another solution?


    I hope this will not consume your time. I apologize for the inconvenience, but I
    need these two classes in order to finish my diploma thesis, and I don't know from who I should ask for help.


    Thank you very much in advance,
    Sofia Georgiakaki
    undergraduate student
    department of Electronic & Computer Engineering
    Technical University of Crete, Greece


    --
    Harsh J
  • Sofia Georgiakaki at Aug 3, 2011 at 4:42 pm
    Thank you for your reply.

    This is what the creator of the patch also recommended.
    The problem is, that I already have developed the project using the new API (I didn't know about the problems), so it won't be so easy to convert
    the whole job. In addition, I'm nervous wondering if the code will run
    after these changes... Aren't those classes in the old API deprecated? If I should apply a patch to deprecate them, it would not be a solution, since the code will be tested on the cluster at my university and I could not apply such a patch there, I suppose.

    In addition, the cluster is possible that
    it will be updated to Hadoop 0.20.203. Will I have a problem using the old api then??


    Hadoop is confusing, I say.

    Thank you,
    Sofia Georgiakaki
  • Janarthanan, Maheshwaran (CDS - San Bruno) at Aug 3, 2011 at 5:25 pm
    Please unsubscribe me.

    -----Original Message-----
    From: Sofia Georgiakaki
    Sent: Wednesday, August 03, 2011 9:42 AM
    To: common-user@hadoop.apache.org
    Subject: Re: TotalOrderPartitioner with new api - help

    Thank you for your reply.

    This is what the creator of the patch also recommended.
    The problem is, that I already have developed the project using the new API (I didn't know about the problems), so it won't be so easy to convert the whole job. In addition, I'm nervous wondering if the code will run after these changes... Aren't those classes in the old API deprecated? If I should apply a patch to deprecate them, it would not be a solution, since the code will be tested on the cluster at my university and I could not apply such a patch there, I suppose.

    In addition, the cluster is possible that it will be updated to Hadoop 0.20.203. Will I have a problem using the old api then??


    Hadoop is confusing, I say.

    Thank you,
    Sofia Georgiakaki

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-user @
categorieshadoop
postedAug 3, '11 at 12:36p
activeAug 3, '11 at 5:25p
posts4
users3
websitehadoop.apache.org...
irc#hadoop

People

Translate

site design / logo © 2022 Grokbase