FAQ
Hi,
I had asked a question about predicting map times in hadoop.
Thanks a lot for the encouraging response.
I want to know if anybody has a code or any idea on how to calculate the
execution time? I mean a small estimation.q


--
Regards,
R.V.

Search Discussions

  • Viraj Bhat at Apr 21, 2011 at 5:52 am
    Hi RV,
    We have literature which does this in research from the University of Washington. Pig is planning to use this framework, in its progress indicators.
    The below papers give you some idea of how to do this estimation.

    http://www.cs.washington.edu/education/courses/cse544/11wi/examples/abe-kristi.pdf

    ftp://ftp.cs.washington.edu/tr/2009/07/UW-CSE-09-07-01.PDF

    Regards
    Viraj

    -----Original Message-----
    From: real great..
    Sent: Wednesday, April 20, 2011 2:29 AM
    To: common-user
    Subject: Execution time.

    Hi,
    I had asked a question about predicting map times in hadoop.
    Thanks a lot for the encouraging response.
    I want to know if anybody has a code or any idea on how to calculate the
    execution time? I mean a small estimation.q


    --
    Regards,
    R.V.
  • Steve Loughran at Apr 26, 2011 at 10:55 am

    On 20/04/11 10:28, real great.. wrote:
    Hi,
    I had asked a question about predicting map times in hadoop.
    Thanks a lot for the encouraging response.
    I want to know if anybody has a code or any idea on how to calculate the
    execution time? I mean a small estimation.q
    1. Surely this is what your final year project should do? It's one thing
    to provide pointers, but to give you code, isn't that, well, against the
    rules.

    2. If you read the papers, the key finding is you can't estimate
    execution time except from historical data or some subsampling. You will
    need to write code to do one of these
  • Real great.. at Apr 26, 2011 at 1:16 pm
    Thanks a lot.I have managed to do it.
    And my final year project is on power aware Hadoop. i do realise its against
    ethics to get the code that way..:)
    On Tue, Apr 26, 2011 at 4:24 PM, Steve Loughran wrote:
    On 20/04/11 10:28, real great.. wrote:

    Hi,
    I had asked a question about predicting map times in hadoop.
    Thanks a lot for the encouraging response.
    I want to know if anybody has a code or any idea on how to calculate the
    execution time? I mean a small estimation.q

    1. Surely this is what your final year project should do? It's one thing to
    provide pointers, but to give you code, isn't that, well, against the rules.

    2. If you read the papers, the key finding is you can't estimate execution
    time except from historical data or some subsampling. You will need to write
    code to do one of these


    --
    Regards,
    R.V.
  • Steve Loughran at Apr 27, 2011 at 9:59 am

    On 26/04/11 14:16, real great.. wrote:
    Thanks a lot.I have managed to do it.
    And my final year project is on power aware Hadoop. i do realise its against
    ethics to get the code that way..:)
    Good.

    What do you mean by power aware

    -awareness of the topology of UPS sources inside a datacentre
    -awareness of CPU voltage level/power drain to schedule work where CPUs
    are capable of being most efficiently used, rather than scheduling work
    on a CPU that will have to ramp up to its full voltage and so be least
    efficient?

    either would be interesting. You could use the existing rack topology
    scripts for UPS topology, but really there should be two topologies, as
    it's block placement where you need the UPS topology
  • Real great.. at Apr 27, 2011 at 3:00 pm
    Am concentrating on block placement strategy. And the second point is what
    am involved in.
    On Wed, Apr 27, 2011 at 3:28 PM, Steve Loughran wrote:
    On 26/1 14:16, real great.. wrote:

    Thanks a lot.I have managed to do it.
    And my final year project is on power aware Hadoop. i do realise its
    against
    ethics to get the code that way..:)
    Good.

    What do you mean by power aware

    -awareness of the topology of UPS sources inside a datacentre
    -awareness of CPU voltage level/power drain to schedule work where CPUs are
    capable of being most efficiently used, rather than scheduling work on a CPU
    that will have to ramp up to its full voltage and so be least efficient?

    either would be interesting. You could use the existing rack topology
    scripts for UPS topology, but really there should be two topologies, as it's
    block placement where you need the UPS topology


    --
    Regards,
    R.V.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-user @
categorieshadoop
postedApr 20, '11 at 9:29a
activeApr 27, '11 at 3:00p
posts6
users3
websitehadoop.apache.org...
irc#hadoop

People

Translate

site design / logo © 2022 Grokbase