FAQ
Hadoop Committers,

The Hadoop project has grown to become a critical platform for many
companies. As we approach a 1.0 release (perhaps within the next 6
months) I think it is time we adopt some more rigor around designing
and testing new features.

As committers, we are already given a couple duties that relate to
quality. From http://wiki.apache.org/hadoop/HowToCommit :
If a committer reviews a patch they've not authored, and believe it
to be of sufficient quality, then they can commit the patch,
otherwise the patch should be cancelled with a clear explanation for
why it was rejected.

...

Patches should be rejected which do not adhere to the guidelines in http://wiki.apache.org/hadoop/HowToContribute
and to the http://wiki.apache.org/hadoop/CodeReviewChecklist.
Committers should always be polite to contributors and try to
instruct and encourage them to contribute better patches.
These have been a good starting point, but I'd like to propose more
rigor.

I propose that before we commit issues marked as "New Feature", they
must have:
1. a design doc attachment
2. a test plan attachment
(Templates to be provided for both)

I believe these 2 items will enhance the communication, design, and
quality of our new features.

Are committers open to adopting these further guidelines?

Thanks,
Nige

Search Discussions

  • Peter Veentjer at Nov 20, 2008 at 6:48 am
    What about thread safety?

    I see that the code base is very large and it is quite hard to analyse
    because there is not much documentation if classes are used by
    multiple threads/single thread/immutable
    Other things I see are:
    - eating up interrupted exceptions
    - very coarse grained locking
    - locking on objects you don't own and nested locking (both deadlock prone)
    - using non threadsafe collections in combination with custom locking
    (doesn't need to be a bad thing, but not by default)
    - java memory model problems (and they are hard to analyse because of
    the lack of documentation)

    These are a few of the things I have found so far.

    I think these things can be simplified by adding them to the guidelines.
    On Thu, Nov 20, 2008 at 7:23 AM, Nigel Daley wrote:
    Hadoop Committers,

    The Hadoop project has grown to become a critical platform for many
    companies. As we approach a 1.0 release (perhaps within the next 6 months)
    I think it is time we adopt some more rigor around designing and testing new
    features.

    As committers, we are already given a couple duties that relate to quality.
    From http://wiki.apache.org/hadoop/HowToCommit :
    If a committer reviews a patch they've not authored, and believe it to be
    of sufficient quality, then they can commit the patch, otherwise the patch
    should be cancelled with a clear explanation for why it was rejected.

    ...

    Patches should be rejected which do not adhere to the guidelines in
    http://wiki.apache.org/hadoop/HowToContribute and to the
    http://wiki.apache.org/hadoop/CodeReviewChecklist. Committers should always
    be polite to contributors and try to instruct and encourage them to
    contribute better patches.
    These have been a good starting point, but I'd like to propose more rigor.

    I propose that before we commit issues marked as "New Feature", they must
    have:
    1. a design doc attachment
    2. a test plan attachment
    (Templates to be provided for both)

    I believe these 2 items will enhance the communication, design, and quality
    of our new features.

    Are committers open to adopting these further guidelines?

    Thanks,
    Nige
  • Nigel Daley at Nov 20, 2008 at 7:19 am
    Thanks Peter, but code review guidelines are a different discussion.
    I'm asking for commitment to feature design docs and test plans in
    this thread.

    Nige
    On Nov 19, 2008, at 10:47 PM, Peter Veentjer wrote:

    What about thread safety?

    I see that the code base is very large and it is quite hard to analyse
    because there is not much documentation if classes are used by
    multiple threads/single thread/immutable
    Other things I see are:
    - eating up interrupted exceptions
    - very coarse grained locking
    - locking on objects you don't own and nested locking (both deadlock
    prone)
    - using non threadsafe collections in combination with custom locking
    (doesn't need to be a bad thing, but not by default)
    - java memory model problems (and they are hard to analyse because of
    the lack of documentation)

    These are a few of the things I have found so far.

    I think these things can be simplified by adding them to the
    guidelines.
    On Thu, Nov 20, 2008 at 7:23 AM, Nigel Daley wrote:
    Hadoop Committers,

    The Hadoop project has grown to become a critical platform for many
    companies. As we approach a 1.0 release (perhaps within the next 6
    months)
    I think it is time we adopt some more rigor around designing and
    testing new
    features.

    As committers, we are already given a couple duties that relate to
    quality.
    From http://wiki.apache.org/hadoop/HowToCommit :
    If a committer reviews a patch they've not authored, and believe
    it to be
    of sufficient quality, then they can commit the patch, otherwise
    the patch
    should be cancelled with a clear explanation for why it was
    rejected.

    ...

    Patches should be rejected which do not adhere to the guidelines in
    http://wiki.apache.org/hadoop/HowToContribute and to the
    http://wiki.apache.org/hadoop/CodeReviewChecklist. Committers
    should always
    be polite to contributors and try to instruct and encourage them to
    contribute better patches.
    These have been a good starting point, but I'd like to propose more
    rigor.

    I propose that before we commit issues marked as "New Feature",
    they must
    have:
    1. a design doc attachment
    2. a test plan attachment
    (Templates to be provided for both)

    I believe these 2 items will enhance the communication, design, and
    quality
    of our new features.

    Are committers open to adopting these further guidelines?

    Thanks,
    Nige
  • Konstantin Shvachko at Nov 20, 2008 at 6:31 pm
    +1 to require design docs and test plans for new features.
    --Konstantin

    Nigel Daley wrote:
    Hadoop Committers,

    The Hadoop project has grown to become a critical platform for many
    companies. As we approach a 1.0 release (perhaps within the next 6
    months) I think it is time we adopt some more rigor around designing and
    testing new features.

    As committers, we are already given a couple duties that relate to
    quality. From http://wiki.apache.org/hadoop/HowToCommit :
    If a committer reviews a patch they've not authored, and believe it to
    be of sufficient quality, then they can commit the patch, otherwise
    the patch should be cancelled with a clear explanation for why it was
    rejected.

    ...

    Patches should be rejected which do not adhere to the guidelines in
    http://wiki.apache.org/hadoop/HowToContribute and to the
    http://wiki.apache.org/hadoop/CodeReviewChecklist. Committers should
    always be polite to contributors and try to instruct and encourage
    them to contribute better patches.
    These have been a good starting point, but I'd like to propose more rigor.

    I propose that before we commit issues marked as "New Feature", they
    must have:
    1. a design doc attachment
    2. a test plan attachment
    (Templates to be provided for both)

    I believe these 2 items will enhance the communication, design, and
    quality of our new features.

    Are committers open to adopting these further guidelines?

    Thanks,
    Nige
  • Doug Cutting at Nov 20, 2008 at 6:55 pm

    Nigel Daley wrote:
    I propose that before we commit issues marked as "New Feature", they
    must have:
    1. a design doc attachment
    2. a test plan attachment
    (Templates to be provided for both)
    Clarifying test expectations is a good idea in principle. I gather that
    what you're after in (1) is an enumeration of new functionality and in
    (2) corresponding tests. That sounds reasonable for large new features,
    but might be overkill for small features, no?

    I'd like to see the templates and some examples. Documents would
    preferably be in some open format, like HTML or plain text.

    Is there an assumption that, before the patch is committed, that the
    tests in the plan will be implemented and passed? Are these to be
    automated tests, with the goal that we'll end up with automated
    regression tests for all features?

    Doug
  • Konstantin Shvachko at Nov 20, 2008 at 8:42 pm

    Is there an assumption that, before the patch is committed, that the
    tests in the plan will be implemented and passed?
    Are these to be
    automated tests, with the goal that we'll end up with automated
    regression tests for all features?
    1. I think we should have a requirement to have all tests that could be
    automated be automated and implemented as JUnit tests.
    Some tests cannot be automated and may require manual testing or testing
    on a large scale, which developers may not have access to.

    2. Therefore, we may have to commit before the entire test plan is completed.
    But the plan should be developed.

    Konstantin

    Doug Cutting wrote:
    Nigel Daley wrote:
    I propose that before we commit issues marked as "New Feature", they
    must have:
    1. a design doc attachment
    2. a test plan attachment
    (Templates to be provided for both)
    Clarifying test expectations is a good idea in principle. I gather that
    what you're after in (1) is an enumeration of new functionality and in
    (2) corresponding tests. That sounds reasonable for large new features,
    but might be overkill for small features, no?

    I'd like to see the templates and some examples. Documents would
    preferably be in some open format, like HTML or plain text.

    Is there an assumption that, before the patch is committed, that the
    tests in the plan will be implemented and passed? Are these to be
    automated tests, with the goal that we'll end up with automated
    regression tests for all features?

    Doug

  • Nigel Daley at Nov 21, 2008 at 12:45 am

    On Nov 20, 2008, at 10:54 AM, Doug Cutting wrote:

    Nigel Daley wrote:
    I propose that before we commit issues marked as "New Feature",
    they must have:
    1. a design doc attachment
    2. a test plan attachment
    (Templates to be provided for both)
    Clarifying test expectations is a good idea in principle. I gather
    that what you're after in (1) is an enumeration of new functionality
    Yes, how the new feature interacts within the system and how to
    interact with it from outside the system (if applicable).
    and in (2) corresponding tests. That sounds reasonable for large
    new features, but might be overkill for small features, no?
    I agree with Arun: "+1 with the expectation that common sense will
    prevail for smaller features". I'll come up with some examples.
    I'd like to see the templates and some examples. Documents would
    preferably be in some open format, like HTML or plain text.
    Ya, I'll work on these next if we have agreement in principle.
    Is there an assumption that, before the patch is committed, that the
    tests in the plan will be implemented and passed?
    Yes, for the automatable tests.
    Are these to be automated tests, with the goal that we'll end up
    with automated regression tests for all features?
    I think all tests should be enumerated in the test plan (positive and
    negative functional, system, scale, performance, etc) with the
    expectation that the positive and negative functional tests will be
    automated with JUnit.

    Nige
  • Nigel Daley at Feb 18, 2009 at 6:58 am
    [resurrecting an old thread now that I've had time to put together a
    proposal]

    Sounds like folks are agreeing we need design docs and test plans for
    features, with common sense prevailing on when to apply this rule to
    smaller features.

    So, let me propose some design doc and test plan templates.

    Design Doc Template:
    Problem Definition
    what problem does the feature solve?
    why is it needed?
    Use Cases
    sequence of simple steps on how users interacts with the feature
    Interaction with Current System
    does this feature build on an existing feature? if so, how?
    how does the user interact with the new feature from outside the system
    how does the new feature interact within the system
    Requirements
    focus on what functionality must exist and minimize discussion of how
    that functionality will be implemented
    list items out of scope
    Design
    give a high level description of the implementation
    algorithms and/or data-structures involved, this may be in the form of
    pseudo-code
    failure analysis
    Future Work
    what additional work is related or could be done, but is not included
    in this design

    Test Plan Template:
    Feature(s) Tested: enumerate the feature(s)
    which Jira issue(s)?
    what is the feature?
    what is the externally visible view of the feature?
    Risk Scenarios: enumerate the bad things that could happen in the
    system that either:
    could be caused by the feature
    could have an effect on the feature
    Test Cases: enumerate all tests in tables
    using table headings Id, Type of Test, Description, Expected Behavior,
    Is Automated?
    enumerate positive and negative test case types
    include functional, system, scale, and performance test case types
    indicate which tests are automated in JUnit suite

    Past Design Doc Examples
    http://issues.apache.org/jira/secure/attachment/12348296/DFSUpgradeProposal3.html

    Past Test Plan Examples
    https://issues.apache.org/jira/secure/attachment/12373559/PermissionsTestPlan2.pdf
    https://issues.apache.org/jira/secure/attachment/12363605/BlockCrcFeatureTestPlan.pdf
    https://issues.apache.org/jira/secure/attachment/12351299/TestPlan-HdfsUpgrade.html

    Some examples requiring the above:
    HADOOP-1700 (file append)
    HADOOP-702 (DFS upgrades)
    HADOOP-3938 (disk quotas)
    HADOOP-372 (input format per dir)
    HADOOP-153 (skip bad records)


    Thoughts? Is this something we can all commit to do?

    Cheers,
    Nige

    Hadoop Committers,

    The Hadoop project has grown to become a critical platform for many
    companies. As we approach a 1.0 release (perhaps within the next 6
    months) I think it is time we adopt some more rigor around designing
    and testing new features.

    As committers, we are already given a couple duties that relate to
    quality. Fromhttp://wiki.apache.org/hadoop/HowToCommit :
    If a committer reviews a patch they've not authored, and believe it
    to be of sufficient quality, then they can commit the patch,
    otherwise the patch should be cancelled with a clear explanation
    for why it was rejected.

    ...

    Patches should be rejected which do not adhere to the guidelines inhttp://wiki.apache.org/hadoop/HowToContribute
    and to thehttp://wiki.apache.org/hadoop/CodeReviewChecklist.
    Committers should always be polite to contributors and try to
    instruct and encourage them to contribute better patches.
    These have been a good starting point, but I'd like to propose more
    rigor.

    I propose that before we commit issues marked as "New Feature", they
    must have:
    1. a design doc attachment
    2. a test plan attachment
    (Templates to be provided for both)

    I believe these 2 items will enhance the communication, design, and
    quality of our new features.

    Are committers open to adopting these further guidelines?

    Thanks,
    Nige
  • Nigel Daley at Feb 18, 2009 at 7:03 am
    [let's see if this formatting comes through better...]

    [resurrecting an old thread now that I've had time to put together a
    proposal]

    Sounds like folks are agreeing we need design docs and test plans for
    features, with common sense prevailing on when to apply this rule to
    smaller features.

    So, let me propose some design doc and test plan templates.

    Design Doc Template:
    • Problem Definition
    • what problem does the feature solve?
    • why is it needed?
    • Use Cases
    • sequence of simple steps on how users interacts with the feature
    • Interaction with Current System
    • does this feature build on an existing feature? if so, how?
    • how does the user interact with the new feature from outside the
    system
    • how does the new feature interact within the system
    • Requirements
    • focus on what functionality must exist and minimize discussion of
    how that functionality will be implemented
    • list items out of scope
    • Design
    • give a high level description of the implementation

    algorithms and/or data-structures involved, this may be in the form of
    pseudo-code
    • failure analysis
    • Future Work
    • what additional work is related or could be done, but is not
    included in this design

    Test Plan Template:
    • Feature(s) Tested: enumerate the feature(s)
    • which Jira issue(s)?
    • what is the feature?
    • what is the externally visible view of the feature?
    • Risk Scenarios: enumerate the bad things that could happen in the
    system that either:
    • could be caused by the feature
    • could have an effect on the feature
    • Test Cases: enumerate all tests in tables
    • using table headings Id, Type of Test, Description, Expected
    Behavior, Is Automated?
    • enumerate positive and negative test case types
    • include functional, system, scale, and performance test case types
    • indicate which tests are automated in JUnit suite

    Past Design Doc Examples
    http://issues.apache.org/jira/secure/attachment/12348296/DFSUpgradeProposal3.html

    Past Test Plan Examples
    https://issues.apache.org/jira/secure/attachment/12373559/PermissionsTestPlan2.pdf
    https://issues.apache.org/jira/secure/attachment/12363605/BlockCrcFeatureTestPlan.pdf
    https://issues.apache.org/jira/secure/attachment/12351299/TestPlan-HdfsUpgrade.html

    Some examples requiring the above:
    • HADOOP-1700 (file append)
    • HADOOP-702 (DFS upgrades)
    • HADOOP-3938 (disk quotas)
    • HADOOP-372 (input format per dir)
    • HADOOP-153 (skip bad records)


    Thoughts? Is this something we can all commit to do?

    Cheers,
    Nige

    Hadoop Committers,

    The Hadoop project has grown to become a critical platform for many
    companies. As we approach a 1.0 release (perhaps within the next 6
    months) I think it is time we adopt some more rigor around designing
    and testing new features.

    As committers, we are already given a couple duties that relate to
    quality. Fromhttp://wiki.apache.org/hadoop/HowToCommit :
    If a committer reviews a patch they've not authored, and believe it
    to be of sufficient quality, then they can commit the patch,
    otherwise the patch should be cancelled with a clear explanation
    for why it was rejected.

    ...

    Patches should be rejected which do not adhere to the guidelines inhttp://wiki.apache.org/hadoop/HowToContribute
    and to thehttp://wiki.apache.org/hadoop/CodeReviewChecklist.
    Committers should always be polite to contributors and try to
    instruct and encourage them to contribute better patches.
    These have been a good starting point, but I'd like to propose more
    rigor.

    I propose that before we commit issues marked as "New Feature", they
    must have:
    1. a design doc attachment
    2. a test plan attachment
    (Templates to be provided for both)

    I believe these 2 items will enhance the communication, design, and
    quality of our new features.

    Are committers open to adopting these further guidelines?

    Thanks,
    Nige
  • Dhruba Borthakur at Feb 18, 2009 at 7:05 am
    +1.

    I would write a design doc and a test plan according to the above guidelines
    for an upcoming new feature "symbolic links" via HADOOP-4044.

    thanks,
    dhruba
    On Tue, Feb 17, 2009 at 11:02 PM, Nigel Daley wrote:

    [let's see if this formatting comes through better...]


    [resurrecting an old thread now that I've had time to put together a
    proposal]

    Sounds like folks are agreeing we need design docs and test plans for
    features, with common sense prevailing on when to apply this rule to smaller
    features.

    So, let me propose some design doc and test plan templates.

    Design Doc Template:
    • Problem Definition
    • what problem does the feature solve?
    • why is it needed?
    • Use Cases
    • sequence of simple steps on how users interacts with the
    feature
    • Interaction with Current System
    • does this feature build on an existing feature? if so,
    how?
    • how does the user interact with the new feature from
    outside the system
    • how does the new feature interact within the system
    • Requirements
    • focus on what functionality must exist and minimize
    discussion of how that functionality will be implemented
    • list items out of scope
    • Design
    • give a high level description of the implementation

    algorithms and/or data-structures involved, this may be in the form of
    pseudo-code
    • failure analysis
    • Future Work
    • what additional work is related or could be done, but is
    not included in this design

    Test Plan Template:
    • Feature(s) Tested: enumerate the feature(s)
    • which Jira issue(s)?
    • what is the feature?
    • what is the externally visible view of the feature?
    • Risk Scenarios: enumerate the bad things that could happen in the
    system that either:
    • could be caused by the feature
    • could have an effect on the feature
    • Test Cases: enumerate all tests in tables
    • using table headings Id, Type of Test, Description,
    Expected Behavior, Is Automated?
    • enumerate positive and negative test case types
    • include functional, system, scale, and performance test
    case types
    • indicate which tests are automated in JUnit suite

    Past Design Doc Examples

    http://issues.apache.org/jira/secure/attachment/12348296/DFSUpgradeProposal3.html

    Past Test Plan Examples

    https://issues.apache.org/jira/secure/attachment/12373559/PermissionsTestPlan2.pdf

    https://issues.apache.org/jira/secure/attachment/12363605/BlockCrcFeatureTestPlan.pdf

    https://issues.apache.org/jira/secure/attachment/12351299/TestPlan-HdfsUpgrade.html

    Some examples requiring the above:
    • HADOOP-1700 (file append)
    • HADOOP-702 (DFS upgrades)
    • HADOOP-3938 (disk quotas)
    • HADOOP-372 (input format per dir)
    • HADOOP-153 (skip bad records)


    Thoughts? Is this something we can all commit to do?

    Cheers,
    Nige


    Hadoop Committers,
    The Hadoop project has grown to become a critical platform for many
    companies. As we approach a 1.0 release (perhaps within the next 6 months)
    I think it is time we adopt some more rigor around designing and testing new
    features.

    As committers, we are already given a couple duties that relate to
    quality. Fromhttp://wiki.apache.org/hadoop/HowToCommit :

    If a committer reviews a patch they've not authored, and believe it to be
    of sufficient quality, then they can commit the patch, otherwise the patch
    should be cancelled with a clear explanation for why it was rejected.

    ...

    Patches should be rejected which do not adhere to the guidelines
    inhttp://wiki.apache.org/hadoop/HowToContribute and to thehttp://
    wiki.apache.org/hadoop/CodeReviewChecklist. Committers should always be
    polite to contributors and try to instruct and encourage them to contribute
    better patches.

    These have been a good starting point, but I'd like to propose more
    rigor.

    I propose that before we commit issues marked as "New Feature", they must
    have:
    1. a design doc attachment
    2. a test plan attachment
    (Templates to be provided for both)

    I believe these 2 items will enhance the communication, design, and
    quality of our new features.

    Are committers open to adopting these further guidelines?

    Thanks,
    Nige
  • Arun C Murthy at Nov 20, 2008 at 9:15 pm

    On Nov 19, 2008, at 10:23 PM, Nigel Daley wrote:
    I propose that before we commit issues marked as "New Feature", they
    must have:
    1. a design doc attachment
    2. a test plan attachment
    (Templates to be provided for both)

    I believe these 2 items will enhance the communication, design, and
    quality of our new features.

    Are committers open to adopting these further guidelines?
    +1 with the expectation that common sense will prevail for smaller
    features. (Maybe we should have a jira-type for "Big Feature" ? *smile*)

    Nigel, do you have a proposal for the templates?

    Maybe it would be good to have a simple textual template with the
    happy implication that people can put the design up as a jira comment
    (with markup?) after the discussion (comments) along with an attachment.
    Implication: it is easier for a late gate-crasher to ignore
    discussions and concentrate on the the 'proposal' as it evolves over
    time.

    Arun

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-dev @
categorieshadoop
postedNov 20, '08 at 6:26a
activeFeb 18, '09 at 7:05a
posts11
users6
websitehadoop.apache.org...
irc#hadoop

People

Translate

site design / logo © 2022 Grokbase