FAQ
Hi,

How long is Hadoop full unit test suit expected to run?
How do you go about running Hadoop tests? I found that it can take hours for
[ant test] target to run which does not seem to be very efficient for
development.
Is there anything I can do to speed up tests (like running Hadoop in a real
cluster)?

Say I would like to fix a bug in Hadoop in ABC.java. Is it OK if I execute
just ABCTest.java (if available) for the development phase before the patch
is attached to JIRA ticket? I don't expect the answer to this question is
positive but I can not think of better workaround for now...

Regards,
Lukas

Search Discussions

  • Enis Soztutar at Jun 17, 2008 at 8:26 am

    Lukas Vlcek wrote:
    Hi,

    How long is Hadoop full unit test suit expected to run?
    How do you go about running Hadoop tests? I found that it can take hours for
    [ant test] target to run which does not seem to be very efficient for
    development.
    Is there anything I can do to speed up tests (like running Hadoop in a real
    cluster)?
    Hi,

    Yes, ant test can take up to an hour. On my machine it completes in less
    than an hour. Previously work has been done to reduce the time that
    tests take, but some of the tests take a long time by nature such as
    testing dfs balance, etc.
    There is an issue to implement ant test-core as a mapred job so that it
    can be submitted to a cluster. That would help a lot.
    Say I would like to fix a bug in Hadoop in ABC.java. Is it OK if I execute
    just ABCTest.java (if available) for the development phase before the patch
    is attached to JIRA ticket? I don't expect the answer to this question is
    positive but I can not think of better workaround for now...
    This very much depends on the patch. If you are "convinced" by running
    TestABC, that the patch would be OK, then you can go ahead and submit
    patch to hudson for QA testing. However, since the resources at Hudson
    is limited, please do not use it for "regular" tests. As a side note,
    please run ant test-patch to check your patch.

    http://wiki.apache.org/hadoop/HowToContribute
    http://wiki.apache.org/hadoop/CodeReviewChecklist

    Enis
    Regards,
    Lukas
  • Lukas Vlcek at Jun 17, 2008 at 9:56 am
    Hi,

    I am aware of HowToContrib wiki page but in my case [ant test] takes more
    then one hour. I can not tell you how much time it takes because I always
    stopped it after 4-5 hours...

    I was running these test on notebook Dell, dual core 1GB of RAM, Windows XP.
    I haven't tried it now after switching to Ubuntu but I think this should not
    make a big difference. What kind of HW are you using for Hadoop testing? I
    would definitely appreciate if [ant test] runs under an hour.

    Regards,
    Lukas
    On Tue, Jun 17, 2008 at 10:26 AM, Enis Soztutar wrote:


    Lukas Vlcek wrote:
    Hi,

    How long is Hadoop full unit test suit expected to run?
    How do you go about running Hadoop tests? I found that it can take hours
    for
    [ant test] target to run which does not seem to be very efficient for
    development.
    Is there anything I can do to speed up tests (like running Hadoop in a
    real
    cluster)?
    Hi,

    Yes, ant test can take up to an hour. On my machine it completes in less
    than an hour. Previously work has been done to reduce the time that tests
    take, but some of the tests take a long time by nature such as testing dfs
    balance, etc.
    There is an issue to implement ant test-core as a mapred job so that it can
    be submitted to a cluster. That would help a lot.
    Say I would like to fix a bug in Hadoop in ABC.java. Is it OK if I execute
    just ABCTest.java (if available) for the development phase before the
    patch
    is attached to JIRA ticket? I don't expect the answer to this question is
    positive but I can not think of better workaround for now...
    This very much depends on the patch. If you are "convinced" by running
    TestABC, that the patch would be OK, then you can go ahead and submit patch
    to hudson for QA testing. However, since the resources at Hudson is limited,
    please do not use it for "regular" tests. As a side note, please run ant
    test-patch to check your patch.

    http://wiki.apache.org/hadoop/HowToContribute
    http://wiki.apache.org/hadoop/CodeReviewChecklist

    Enis
    Regards,
    Lukas


    --
    http://blog.lukas-vlcek.com/
  • Enis Soztutar at Jun 17, 2008 at 3:35 pm
    Just standard box like yours (ubuntu). I suspect something must be
    wrong. Could you determine which tests take long time. On hudson, it
    seems that tests take 1:30 on average.
    http://hudson.zones.apache.org/hudson/view/Hadoop/


    Lukas Vlcek wrote:
    Hi,

    I am aware of HowToContrib wiki page but in my case [ant test] takes more
    then one hour. I can not tell you how much time it takes because I always
    stopped it after 4-5 hours...

    I was running these test on notebook Dell, dual core 1GB of RAM, Windows XP.
    I haven't tried it now after switching to Ubuntu but I think this should not
    make a big difference. What kind of HW are you using for Hadoop testing? I
    would definitely appreciate if [ant test] runs under an hour.

    Regards,
    Lukas

    On Tue, Jun 17, 2008 at 10:26 AM, Enis Soztutar wrote:

    Lukas Vlcek wrote:

    Hi,

    How long is Hadoop full unit test suit expected to run?
    How do you go about running Hadoop tests? I found that it can take hours
    for
    [ant test] target to run which does not seem to be very efficient for
    development.
    Is there anything I can do to speed up tests (like running Hadoop in a
    real
    cluster)?

    Hi,

    Yes, ant test can take up to an hour. On my machine it completes in less
    than an hour. Previously work has been done to reduce the time that tests
    take, but some of the tests take a long time by nature such as testing dfs
    balance, etc.
    There is an issue to implement ant test-core as a mapred job so that it can
    be submitted to a cluster. That would help a lot.

    Say I would like to fix a bug in Hadoop in ABC.java. Is it OK if I execute
    just ABCTest.java (if available) for the development phase before the
    patch
    is attached to JIRA ticket? I don't expect the answer to this question is
    positive but I can not think of better workaround for now...

    This very much depends on the patch. If you are "convinced" by running
    TestABC, that the patch would be OK, then you can go ahead and submit patch
    to hudson for QA testing. However, since the resources at Hudson is limited,
    please do not use it for "regular" tests. As a side note, please run ant
    test-patch to check your patch.

    http://wiki.apache.org/hadoop/HowToContribute
    http://wiki.apache.org/hadoop/CodeReviewChecklist

    Enis

    Regards,
    Lukas


Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-user @
categorieshadoop
postedJun 17, '08 at 7:03a
activeJun 17, '08 at 3:35p
posts4
users2
websitehadoop.apache.org...
irc#hadoop

2 users in discussion

Enis Soztutar: 2 posts Lukas Vlcek: 2 posts

People

Translate

site design / logo © 2022 Grokbase