FAQ
There had a been a very consistent usage of double quotes for all strings
in all parts of bosh repo. Why are these being changed to single quotes? Is
this a high priority?!

Nic

--
Dr Nic Williams
Stark & Wayne LLC - consultancy for Cloud Foundry users
http://drnicwilliams.com
http://starkandwayne.com
cell +1 (415) 860-2185
twitter @drnic

Search Discussions

  • Dr Nic Williams at Jun 24, 2013 at 8:46 pm
    If anyone is looking for low hanging fruit to do that adds immediate value,
    please add inline documentation to methods and classes. That'd be awesome
    as you're learning the code base.
  • Abhi Hiremagalur at Jun 24, 2013 at 9:15 pm

    If anyone is looking for low hanging fruit to do that adds immediate
    value, please add inline documentation to methods and classes. That'd be
    awesome as you're learning the code base.
    Inline documentation for classes/methods intended to be a part of external
    APIs (e.g. CPI methods?) sound great.

    For everything else, I find it most useful to read and run well
    written specs. Hopefully we'll start to see many more of these over time.
  • Abhi Hiremagalur at Jun 24, 2013 at 9:12 pm
    I've been meaning to send an email to the dev list to discuss this,
    apologies for the delay.

    In the absence of an existing set of conventions for the Ruby code in BOSH
    we figured bbatsov's
    styleguide<https://github.com/bbatsov/ruby-style-guide>would be a good
    place to start. We dropped his
    rubocop <https://github.com/bbatsov/rubocop> gem as a tool to help us to
    conform to this styleguide gradually.

    This is isn't a particularly high priority, but it's also relatively cheap
    and, we thought, valuable. Is there a cost associated with this change that
    we haven't accounted for?

    Abhi

    On Mon, Jun 24, 2013 at 12:40 PM, Dr Nic Williams
    wrote:
    There had a been a very consistent usage of double quotes for all strings
    in all parts of bosh repo. Why are these being changed to single quotes? Is
    this a high priority?!

    Nic

    --
    Dr Nic Williams
    Stark & Wayne LLC - consultancy for Cloud Foundry users
    http://drnicwilliams.com
    http://starkandwayne.com
    cell +1 (415) 860-2185
    twitter @drnic
  • Dr Nic Williams at Jun 24, 2013 at 9:31 pm
    Can we use bosh's own source as the basis of the style guide instead? It
    had a very consistent style. Someone else's styleguide could have been
    edited for the existing bosh style.

    Re: "read well written specs" - next time I'm up in SF, I'll come visit and
    you can show me some well written specs that explain how bosh works. I've
    read them and I'm still confused how this beast of a program works :D
    Massive nested Hashes being passed around. Written explanations of each
    Hash being passed in and out would make it much easier to comprehend and
    contribute, I think.

    I'd love to meet the person who created the meme "code should be self
    documenting" and "only learn a codebase by reading specs" and ask them if
    they still agree with it. If they don't, then I'll ask them to repent in
    public :)

    Bosh is a huge code base, with lots of moving parts that aren't easy to run
    locally together. I think more documentation of public & internal aspects
    is a worthy goal to improve the rate of onboarding of new full-time and new
    part-time devs to get into the mindset of the problem that is being solved.
    I am not confident that specs communicate as efficiently as the written
    word.

    Nic


    On Mon, Jun 24, 2013 at 2:11 PM, Abhi Hiremagalur wrote:

    I've been meaning to send an email to the dev list to discuss this,
    apologies for the delay.

    In the absence of an existing set of conventions for the Ruby code in BOSH
    we figured bbatsov's styleguide<https://github.com/bbatsov/ruby-style-guide>would be a good place to start. We dropped his
    rubocop <https://github.com/bbatsov/rubocop> gem as a tool to help us to
    conform to this styleguide gradually.

    This is isn't a particularly high priority, but it's also relatively cheap
    and, we thought, valuable. Is there a cost associated with this change that
    we haven't accounted for?

    Abhi

    On Mon, Jun 24, 2013 at 12:40 PM, Dr Nic Williams <drnicwilliams@gmail.com
    wrote:
    There had a been a very consistent usage of double quotes for all strings
    in all parts of bosh repo. Why are these being changed to single quotes? Is
    this a high priority?!

    Nic

    --
    Dr Nic Williams
    Stark & Wayne LLC - consultancy for Cloud Foundry users
    http://drnicwilliams.com
    http://starkandwayne.com
    cell +1 (415) 860-2185
    twitter @drnic

    --
    Dr Nic Williams
    Stark & Wayne LLC - consultancy for Cloud Foundry users
    http://drnicwilliams.com
    http://starkandwayne.com
    cell +1 (415) 860-2185
    twitter @drnic
  • Dr Nic Williams at Jun 24, 2013 at 9:37 pm
    https://github.com/bbatsov/rubocop/blob/master/config/default.yml - is this
    the config we're using or do we have overrides/changes?

    On Mon, Jun 24, 2013 at 2:30 PM, Dr Nic Williams wrote:

    Can we use bosh's own source as the basis of the style guide instead? It
    had a very consistent style. Someone else's styleguide could have been
    edited for the existing bosh style.

    Re: "read well written specs" - next time I'm up in SF, I'll come visit
    and you can show me some well written specs that explain how bosh works.
    I've read them and I'm still confused how this beast of a program works :D
    Massive nested Hashes being passed around. Written explanations of each
    Hash being passed in and out would make it much easier to comprehend and
    contribute, I think.

    I'd love to meet the person who created the meme "code should be self
    documenting" and "only learn a codebase by reading specs" and ask them if
    they still agree with it. If they don't, then I'll ask them to repent in
    public :)

    Bosh is a huge code base, with lots of moving parts that aren't easy to
    run locally together. I think more documentation of public & internal
    aspects is a worthy goal to improve the rate of onboarding of new full-time
    and new part-time devs to get into the mindset of the problem that is being
    solved. I am not confident that specs communicate as efficiently as the
    written word.

    Nic


    On Mon, Jun 24, 2013 at 2:11 PM, Abhi Hiremagalur wrote:

    I've been meaning to send an email to the dev list to discuss this,
    apologies for the delay.

    In the absence of an existing set of conventions for the Ruby code in
    BOSH we figured bbatsov's styleguide<https://github.com/bbatsov/ruby-style-guide>would be a good place to start. We dropped his
    rubocop <https://github.com/bbatsov/rubocop> gem as a tool to help us to
    conform to this styleguide gradually.

    This is isn't a particularly high priority, but it's also relatively
    cheap and, we thought, valuable. Is there a cost associated with this
    change that we haven't accounted for?

    Abhi

    On Mon, Jun 24, 2013 at 12:40 PM, Dr Nic Williams <
    drnicwilliams@gmail.com> wrote:
    There had a been a very consistent usage of double quotes for all
    strings in all parts of bosh repo. Why are these being changed to single
    quotes? Is this a high priority?!

    Nic

    --
    Dr Nic Williams
    Stark & Wayne LLC - consultancy for Cloud Foundry users
    http://drnicwilliams.com
    http://starkandwayne.com
    cell +1 (415) 860-2185
    twitter @drnic

    --
    Dr Nic Williams
    Stark & Wayne LLC - consultancy for Cloud Foundry users
    http://drnicwilliams.com
    http://starkandwayne.com
    cell +1 (415) 860-2185
    twitter @drnic


    --
    Dr Nic Williams
    Stark & Wayne LLC - consultancy for Cloud Foundry users
    http://drnicwilliams.com
    http://starkandwayne.com
    cell +1 (415) 860-2185
    twitter @drnic
  • Matt Stine at Jun 24, 2013 at 9:52 pm
    Can I inject a "both/and" rather than an "either/or" here?

    I greatly appreciate the idea of well-written, readable specs that document
    the code. Why is this good? Because if they pass, it tells you something
    very useful about how the code works and how the API is to be used.

    But they aren't a panacea. In ideal world, yes, you'd be able to understand
    everything that's happening within an application just by understanding
    it's API. Quite frankly, BOSH's API isn't that good yet. It clearly wasn't
    designed with this philosophy in mind, and it may very well be complex
    enough that this will never be sufficient. As Dr. Nic says, there are just
    so many moving parts...

    I'm afraid we can't afford to wait to find out by incremental addition of
    the type of specs that we're talking about here. Sometime we must place
    pragmatism before idealism.

    In the interim, prose documentation that at least expresses what we believe
    to be true about this system will go a long way toward helping us
    understand what is true. *Then* maybe we can codify that in a spec as well.

    Thoughts?

    On Mon, Jun 24, 2013 at 2:36 PM, Dr Nic Williams wrote:

    https://github.com/bbatsov/rubocop/blob/master/config/default.yml - is
    this the config we're using or do we have overrides/changes?

    On Mon, Jun 24, 2013 at 2:30 PM, Dr Nic Williams wrote:

    Can we use bosh's own source as the basis of the style guide instead? It
    had a very consistent style. Someone else's styleguide could have been
    edited for the existing bosh style.

    Re: "read well written specs" - next time I'm up in SF, I'll come visit
    and you can show me some well written specs that explain how bosh works.
    I've read them and I'm still confused how this beast of a program works :D
    Massive nested Hashes being passed around. Written explanations of each
    Hash being passed in and out would make it much easier to comprehend and
    contribute, I think.

    I'd love to meet the person who created the meme "code should be self
    documenting" and "only learn a codebase by reading specs" and ask them if
    they still agree with it. If they don't, then I'll ask them to repent in
    public :)

    Bosh is a huge code base, with lots of moving parts that aren't easy to
    run locally together. I think more documentation of public & internal
    aspects is a worthy goal to improve the rate of onboarding of new full-time
    and new part-time devs to get into the mindset of the problem that is being
    solved. I am not confident that specs communicate as efficiently as the
    written word.

    Nic


    On Mon, Jun 24, 2013 at 2:11 PM, Abhi Hiremagalur wrote:

    I've been meaning to send an email to the dev list to discuss this,
    apologies for the delay.

    In the absence of an existing set of conventions for the Ruby code in
    BOSH we figured bbatsov's styleguide<https://github.com/bbatsov/ruby-style-guide>would be a good place to start. We dropped his
    rubocop <https://github.com/bbatsov/rubocop> gem as a tool to help us
    to conform to this styleguide gradually.

    This is isn't a particularly high priority, but it's also relatively
    cheap and, we thought, valuable. Is there a cost associated with this
    change that we haven't accounted for?

    Abhi

    On Mon, Jun 24, 2013 at 12:40 PM, Dr Nic Williams <
    drnicwilliams@gmail.com> wrote:
    There had a been a very consistent usage of double quotes for all
    strings in all parts of bosh repo. Why are these being changed to single
    quotes? Is this a high priority?!

    Nic

    --
    Dr Nic Williams
    Stark & Wayne LLC - consultancy for Cloud Foundry users
    http://drnicwilliams.com
    http://starkandwayne.com
    cell +1 (415) 860-2185
    twitter @drnic

    --
    Dr Nic Williams
    Stark & Wayne LLC - consultancy for Cloud Foundry users
    http://drnicwilliams.com
    http://starkandwayne.com
    cell +1 (415) 860-2185
    twitter @drnic


    --
    Dr Nic Williams
    Stark & Wayne LLC - consultancy for Cloud Foundry users
    http://drnicwilliams.com
    http://starkandwayne.com
    cell +1 (415) 860-2185
    twitter @drnic


    --
    Matt Stine | Community Engineer, Cloud Foundry | Pivotal
    901-493-5546 | mstine@gopivotal.com

    goPivotal.com
  • Dr Nic Williams at Jun 24, 2013 at 10:03 pm
    Personally, I find it helpful to write documentation for any method I
    struggled with and ultimately came to understand. A quick PR and shazzam,
    bonus docs!
  • Matthew Sackman at Jun 25, 2013 at 9:48 am

    On Mon, Jun 24, 2013 at 02:52:02PM -0700, Matt Stine wrote:
    In ideal world, yes, you'd be able to understand
    everything that's happening within an application just by understanding
    it's API.
    Any large and complex system cannot be held completely in a programmer's
    mind, no matter how good they are. This goes some way to explain why the
    core components of any large complex system tend to be very small and
    written by a tiny but extremely smart team. For example, I think the
    core dev team of the ESX hypervisor is about 6 people. The postgres
    kernel team is similarly small.

    To help fit more of a system into a programmer's head, the APIs need to
    be complete, unsurprising, well documented, intuitive, unambiguous,
    predictable and reliable. As soon as people have to start digging into
    the source code to try and understand an API, the API has failed.
    In the interim, prose documentation that at least expresses what we believe
    to be true about this system will go a long way toward helping us
    understand what is true. *Then* maybe we can codify that in a spec as well.
    As many of you know, I wrote the first RabbitMQ and have written many of
    the large subsystems of RabbitMQ too - the paging layer and the
    active/active (async) HA layer too. These are not simple things. Rabbit
    is written in Erlang. Erlang has an optional type system which we do
    use. However, there is *no* excuse for not having good prose, for
    example, the long essays at:

    http://hg.rabbitmq.com/rabbitmq-server/file/68d851c51cc3/src/rabbit_variable_queue.erl#l32
    http://hg.rabbitmq.com/rabbitmq-server/file/68d851c51cc3/src/gm.erl#l19
    http://hg.rabbitmq.com/rabbitmq-server/file/68d851c51cc3/src/rabbit_mirror_queue_coordinator.erl#l50
    http://hg.rabbitmq.com/rabbitmq-server/file/68d851c51cc3/src/rabbit_msg_store.erl#l177

    All of those I originally wrote and they have been maintained. There is
    obviously a maintenance burden for having such essays: they *must* be
    kept up to date. But equally, there's a maintenance gain: the purpose of
    such prose is to get people who've not seen that area of code thinking
    in the right way very quickly - the scene has been set, so to speak, for
    the code they are about to encounter. For more experienced developers,
    the prose refreshes their memory when they come back to a module 6
    months later. It very much helps to write such prose as a form of
    finalised brain dump: to note down everything that was in your mind when
    you wrote the code and should be in the mind of all others who tread
    that path.

    Our larger subsystem APIs are often separated out and documented
    independently, for example
    http://hg.rabbitmq.com/rabbitmq-server/file/68d851c51cc3/src/rabbit_backing_queue.erl

    Needless to say, we also have large test suites and we rely on them for
    regression tests.

    Now I'm not a proponent of TDD or even pervasive unit testing. I value
    integration and end-to-end tests for regression spotting, and that's
    about it. I tend to feel you should be able to reason about code and
    reason about your changes to code, and that is worth much more than some
    mocks and some unit tests. To be able to do this requires more of an
    understanding of how components interact with each other and what the
    contracts and invariants are (and consequently, encourages you to
    decouple components, which is always beneficial). To some extent, I feel
    some people think TDD absolves devs of the responsibility of having that
    level of understanding. This is most certainly a falsehood - even with
    TDD and pervasive testing, without that level of understanding, the
    tests will test the wrong thing.

    Complex areas of code which you're not going to be able to test due to
    the difficulty of hitting reliably a certain combination of events can
    only be reasoned about and documented through comments. For example:
    http://hg.rabbitmq.com/rabbitmq-server/file/68d851c51cc3/src/rabbit_msg_store.erl#l551
    client_read3 is an internal function (ultimately called from the public
    function read) when one queue is trying to read a message off disk. But
    there could well be a GC (parallel and concurrent!) of the on-disk store
    involving the file containing the requested message at the same time.
    There are many possible combination of events regarding locking of that
    file, interaction with the GC, the possibility that the message has been
    moved by the GC but only after we first looked up its location, etc etc.
    You just can't test all these cases. So litter the code with comments
    that explain your thinking and hope that the peer code review either
    agrees, or better yet, manages to come up with some other combination of
    events that you've not thought of. If after five years of users using
    your code you never get a bug report in that area, then either the code
    is correct, or it's never used, or it falls in the
    acceptable-defect-rate bucket.

    All that said I may well be speaking from a position of ignorance about
    Ruby and TDD with Ruby, and even more so with Bosh - I've observed one
    day's worth of PP with the backend/services team and that's about it. So
    these comments should certainly be taken in a very general (and probably
    rather ill-informed) sense. My general fears about specs and mocks is
    that people only tend to test the common cases. They test what the
    feature in tracker says the feature should do. There's no negative
    testing, there's no probing of the interesting corner cases, and there
    seems to be little defensive coding in general. A large complex system
    has inevitably a large number of invariants and assumptions that a dev
    and a user needs to be aware of, and these things are seldom well
    captured by specs, IMO. Fify *words* of English which describe what you
    can and cannot do with an API are worth much more to me *as a user* (and
    probably also as a dev) than fifty *lines* of a spec that test one or
    two known good code paths, but don't actually cover what I want to use
    the API for.

    For Rabbit, and our test suites, we try to get as near to 100% code
    coverage as we can. Most modules we're over 90%. For some of the pure
    functional modules, 100% is possible. However, 100% code coverage means
    *bugger all* in the grand scheme of things. 100% code *path* coverage
    would be meaningful, but is impossible to achieve in most cases. For
    some areas, we have gone down the quick-check path. I would encourage
    people to use quick-check and similar tools: they are very useful, and
    go some distance further than just covering lines of code: if you have a
    well written quick-check soak test run a week or more on some code that
    is beyond mentally reasoning about, then you might have some degree of
    belief in the correctness of that code. If you're lucky. There are areas
    of code in Rabbit which are so complex that we have done normal tests,
    quick check tests, we've used the type system, and we've even written
    discrete event simulators to try and prove that the code copes with
    every possible combination of events. But give it two years of users
    banging on it, and bugs will eventually creep out. That's just the
    nature of trying to solve hard problems.

    Apologies if this is all unnecessarily preaching-to-the-converted, or
    just pointlessly ranty. Not my intention at all.

    Matthew
  • Dr Nic Williams at Jun 25, 2013 at 1:37 pm
    Wow, those are some impressive examples of code prose!!
    --
    Dr Nic Williams
    Stark & Wayne LLC - the consultancy for Cloud Foundry
    http://starkandwayne.com
    +1 415 860 2185
    twitter: drnic
    On Tue, Jun 25, 2013 at 2:48 AM, Matthew Sackman wrote:
    On Mon, Jun 24, 2013 at 02:52:02PM -0700, Matt Stine wrote:
    In ideal world, yes, you'd be able to understand
    everything that's happening within an application just by understanding
    it's API.
    Any large and complex system cannot be held completely in a programmer's
    mind, no matter how good they are. This goes some way to explain why the
    core components of any large complex system tend to be very small and
    written by a tiny but extremely smart team. For example, I think the
    core dev team of the ESX hypervisor is about 6 people. The postgres
    kernel team is similarly small.
    To help fit more of a system into a programmer's head, the APIs need to
    be complete, unsurprising, well documented, intuitive, unambiguous,
    predictable and reliable. As soon as people have to start digging into
    the source code to try and understand an API, the API has failed.
    In the interim, prose documentation that at least expresses what we believe
    to be true about this system will go a long way toward helping us
    understand what is true. *Then* maybe we can codify that in a spec as well.
    As many of you know, I wrote the first RabbitMQ and have written many of
    the large subsystems of RabbitMQ too - the paging layer and the
    active/active (async) HA layer too. These are not simple things. Rabbit
    is written in Erlang. Erlang has an optional type system which we do
    use. However, there is *no* excuse for not having good prose, for
    example, the long essays at:
    http://hg.rabbitmq.com/rabbitmq-server/file/68d851c51cc3/src/rabbit_variable_queue.erl#l32
    http://hg.rabbitmq.com/rabbitmq-server/file/68d851c51cc3/src/gm.erl#l19
    http://hg.rabbitmq.com/rabbitmq-server/file/68d851c51cc3/src/rabbit_mirror_queue_coordinator.erl#l50
    http://hg.rabbitmq.com/rabbitmq-server/file/68d851c51cc3/src/rabbit_msg_store.erl#l177
    All of those I originally wrote and they have been maintained. There is
    obviously a maintenance burden for having such essays: they *must* be
    kept up to date. But equally, there's a maintenance gain: the purpose of
    such prose is to get people who've not seen that area of code thinking
    in the right way very quickly - the scene has been set, so to speak, for
    the code they are about to encounter. For more experienced developers,
    the prose refreshes their memory when they come back to a module 6
    months later. It very much helps to write such prose as a form of
    finalised brain dump: to note down everything that was in your mind when
    you wrote the code and should be in the mind of all others who tread
    that path.
    Our larger subsystem APIs are often separated out and documented
    independently, for example
    http://hg.rabbitmq.com/rabbitmq-server/file/68d851c51cc3/src/rabbit_backing_queue.erl
    Needless to say, we also have large test suites and we rely on them for
    regression tests.
    Now I'm not a proponent of TDD or even pervasive unit testing. I value
    integration and end-to-end tests for regression spotting, and that's
    about it. I tend to feel you should be able to reason about code and
    reason about your changes to code, and that is worth much more than some
    mocks and some unit tests. To be able to do this requires more of an
    understanding of how components interact with each other and what the
    contracts and invariants are (and consequently, encourages you to
    decouple components, which is always beneficial). To some extent, I feel
    some people think TDD absolves devs of the responsibility of having that
    level of understanding. This is most certainly a falsehood - even with
    TDD and pervasive testing, without that level of understanding, the
    tests will test the wrong thing.
    Complex areas of code which you're not going to be able to test due to
    the difficulty of hitting reliably a certain combination of events can
    only be reasoned about and documented through comments. For example:
    http://hg.rabbitmq.com/rabbitmq-server/file/68d851c51cc3/src/rabbit_msg_store.erl#l551
    client_read3 is an internal function (ultimately called from the public
    function read) when one queue is trying to read a message off disk. But
    there could well be a GC (parallel and concurrent!) of the on-disk store
    involving the file containing the requested message at the same time.
    There are many possible combination of events regarding locking of that
    file, interaction with the GC, the possibility that the message has been
    moved by the GC but only after we first looked up its location, etc etc.
    You just can't test all these cases. So litter the code with comments
    that explain your thinking and hope that the peer code review either
    agrees, or better yet, manages to come up with some other combination of
    events that you've not thought of. If after five years of users using
    your code you never get a bug report in that area, then either the code
    is correct, or it's never used, or it falls in the
    acceptable-defect-rate bucket.
    All that said I may well be speaking from a position of ignorance about
    Ruby and TDD with Ruby, and even more so with Bosh - I've observed one
    day's worth of PP with the backend/services team and that's about it. So
    these comments should certainly be taken in a very general (and probably
    rather ill-informed) sense. My general fears about specs and mocks is
    that people only tend to test the common cases. They test what the
    feature in tracker says the feature should do. There's no negative
    testing, there's no probing of the interesting corner cases, and there
    seems to be little defensive coding in general. A large complex system
    has inevitably a large number of invariants and assumptions that a dev
    and a user needs to be aware of, and these things are seldom well
    captured by specs, IMO. Fify *words* of English which describe what you
    can and cannot do with an API are worth much more to me *as a user* (and
    probably also as a dev) than fifty *lines* of a spec that test one or
    two known good code paths, but don't actually cover what I want to use
    the API for.
    For Rabbit, and our test suites, we try to get as near to 100% code
    coverage as we can. Most modules we're over 90%. For some of the pure
    functional modules, 100% is possible. However, 100% code coverage means
    *bugger all* in the grand scheme of things. 100% code *path* coverage
    would be meaningful, but is impossible to achieve in most cases. For
    some areas, we have gone down the quick-check path. I would encourage
    people to use quick-check and similar tools: they are very useful, and
    go some distance further than just covering lines of code: if you have a
    well written quick-check soak test run a week or more on some code that
    is beyond mentally reasoning about, then you might have some degree of
    belief in the correctness of that code. If you're lucky. There are areas
    of code in Rabbit which are so complex that we have done normal tests,
    quick check tests, we've used the type system, and we've even written
    discrete event simulators to try and prove that the code copes with
    every possible combination of events. But give it two years of users
    banging on it, and bugs will eventually creep out. That's just the
    nature of trying to solve hard problems.
    Apologies if this is all unnecessarily preaching-to-the-converted, or
    just pointlessly ranty. Not my intention at all.
    Matthew
  • Ferran Rodenas at Jun 25, 2013 at 2:03 pm
    Matt's message itself is an example of prose ;)

    - Ferdy


    2013/6/25 Dr Nic Williams <drnicwilliams@gmail.com>
    Wow, those are some impressive examples of code prose!!
    --
    Dr Nic Williams
    Stark & Wayne LLC - the consultancy for Cloud Foundry
    http://starkandwayne.com
    +1 415 860 2185
    twitter: drnic

    On Tue, Jun 25, 2013 at 2:48 AM, Matthew Sackman wrote:
    On Mon, Jun 24, 2013 at 02:52:02PM -0700, Matt Stine wrote:
    In ideal world, yes, you'd be able to understand
    everything that's happening within an application just by understanding
    it's API.
    Any large and complex system cannot be held completely in a programmer's
    mind, no matter how good they are. This goes some way to explain why the
    core components of any large complex system tend to be very small and
    written by a tiny but extremely smart team. For example, I think the
    core dev team of the ESX hypervisor is about 6 people. The postgres
    kernel team is similarly small.

    To help fit more of a system into a programmer's head, the APIs need to
    be complete, unsurprising, well documented, intuitive, unambiguous,
    predictable and reliable. As soon as people have to start digging into
    the source code to try and understand an API, the API has failed.
    In the interim, prose documentation that at least expresses what we believe
    to be true about this system will go a long way toward helping us
    understand what is true. *Then* maybe we can codify that in a spec as
    well.

    As many of you know, I wrote the first RabbitMQ and have written many of
    the large subsystems of RabbitMQ too - the paging layer and the
    active/active (async) HA layer too. These are not simple things. Rabbit
    is written in Erlang. Erlang has an optional type system which we do
    use. However, there is *no* excuse for not having good prose, for
    example, the long essays at:


    http://hg.rabbitmq.com/rabbitmq-server/file/68d851c51cc3/src/rabbit_variable_queue.erl#l32
    http://hg.rabbitmq.com/rabbitmq-server/file/68d851c51cc3/src/gm.erl#l19

    http://hg.rabbitmq.com/rabbitmq-server/file/68d851c51cc3/src/rabbit_mirror_queue_coordinator.erl#l50

    http://hg.rabbitmq.com/rabbitmq-server/file/68d851c51cc3/src/rabbit_msg_store.erl#l177

    All of those I originally wrote and they have been maintained. There is
    obviously a maintenance burden for having such essays: they *must* be
    kept up to date. But equally, there's a maintenance gain: the purpose of
    such prose is to get people who've not seen that area of code thinking
    in the right way very quickly - the scene has been set, so to speak, for
    the code they are about to encounter. For more experienced developers,
    the prose refreshes their memory when they come back to a module 6
    months later. It very much helps to write such prose as a form of
    finalised brain dump: to note down everything that was in your mind when
    you wrote the code and should be in the mind of all others who tread
    that path.

    Our larger subsystem APIs are often separated out and documented
    independently, for example

    http://hg.rabbitmq.com/rabbitmq-server/file/68d851c51cc3/src/rabbit_backing_queue.erl

    Needless to say, we also have large test suites and we rely on them for
    regression tests.

    Now I'm not a proponent of TDD or even pervasive unit testing. I value
    integration and end-to-end tests for regression spotting, and that's
    about it. I tend to feel you should be able to reason about code and
    reason about your changes to code, and that is worth much more than some
    mocks and some unit tests. To be able to do this requires more of an
    understanding of how components interact with each other and what the
    contracts and invariants are (and consequently, encourages you to
    decouple components, which is always beneficial). To some extent, I feel
    some people think TDD absolves devs of the responsibility of having that
    level of understanding. This is most certainly a falsehood - even with
    TDD and pervasive testing, without that level of understanding, the
    tests will test the wrong thing.

    Complex areas of code which you're not going to be able to test due to
    the difficulty of hitting reliably a certain combination of events can
    only be reasoned about and documented through comments. For example:

    http://hg.rabbitmq.com/rabbitmq-server/file/68d851c51cc3/src/rabbit_msg_store.erl#l551
    client_read3 is an internal function (ultimately called from the public
    function read) when one queue is trying to read a message off disk. But
    there could well be a GC (parallel and concurrent!) of the on-disk store
    involving the file containing the requested message at the same time.
    There are many possible combination of events regarding locking of that
    file, interaction with the GC, the possibility that the message has been
    moved by the GC but only after we first looked up its location, etc etc.
    You just can't test all these cases. So litter the code with comments
    that explain your thinking and hope that the peer code review either
    agrees, or better yet, manages to come up with some other combination of
    events that you've not thought of. If after five years of users using
    your code you never get a bug report in that area, then either the code
    is correct, or it's never used, or it falls in the
    acceptable-defect-rate bucket.

    All that said I may well be speaking from a position of ignorance about
    Ruby and TDD with Ruby, and even more so with Bosh - I've observed one
    day's worth of PP with the backend/services team and that's about it. So
    these comments should certainly be taken in a very general (and probably
    rather ill-informed) sense. My general fears about specs and mocks is
    that people only tend to test the common cases. They test what the
    feature in tracker says the feature should do. There's no negative
    testing, there's no probing of the interesting corner cases, and there
    seems to be little defensive coding in general. A large complex system
    has inevitably a large number of invariants and assumptions that a dev
    and a user needs to be aware of, and these things are seldom well
    captured by specs, IMO. Fify *words* of English which describe what you
    can and cannot do with an API are worth much more to me *as a user* (and
    probably also as a dev) than fifty *lines* of a spec that test one or
    two known good code paths, but don't actually cover what I want to use
    the API for.

    For Rabbit, and our test suites, we try to get as near to 100% code
    coverage as we can. Most modules we're over 90%. For some of the pure
    functional modules, 100% is possible. However, 100% code coverage means
    *bugger all* in the grand scheme of things. 100% code *path* coverage
    would be meaningful, but is impossible to achieve in most cases. For
    some areas, we have gone down the quick-check path. I would encourage
    people to use quick-check and similar tools: they are very useful, and
    go some distance further than just covering lines of code: if you have a
    well written quick-check soak test run a week or more on some code that
    is beyond mentally reasoning about, then you might have some degree of
    belief in the correctness of that code. If you're lucky. There are areas
    of code in Rabbit which are so complex that we have done normal tests,
    quick check tests, we've used the type system, and we've even written
    discrete event simulators to try and prove that the code copes with
    every possible combination of events. But give it two years of users
    banging on it, and bugs will eventually creep out. That's just the
    nature of trying to solve hard problems.

    Apologies if this is all unnecessarily preaching-to-the-converted, or
    just pointlessly ranty. Not my intention at all.

    Matthew
  • Alexis Richardson at Jun 25, 2013 at 2:06 pm
    recursion ftw ;-)
    On Tue, Jun 25, 2013 at 3:03 PM, Ferran Rodenas wrote:
    Matt's message itself is an example of prose ;)

    - Ferdy


    2013/6/25 Dr Nic Williams <drnicwilliams@gmail.com>
    Wow, those are some impressive examples of code prose!!
    --
    Dr Nic Williams
    Stark & Wayne LLC - the consultancy for Cloud Foundry
    http://starkandwayne.com
    +1 415 860 2185
    twitter: drnic


    On Tue, Jun 25, 2013 at 2:48 AM, Matthew Sackman <matthew@wellquite.org>
    wrote:
    On Mon, Jun 24, 2013 at 02:52:02PM -0700, Matt Stine wrote:
    In ideal world, yes, you'd be able to understand
    everything that's happening within an application just by understanding
    it's API.
    Any large and complex system cannot be held completely in a programmer's
    mind, no matter how good they are. This goes some way to explain why the
    core components of any large complex system tend to be very small and
    written by a tiny but extremely smart team. For example, I think the
    core dev team of the ESX hypervisor is about 6 people. The postgres
    kernel team is similarly small.

    To help fit more of a system into a programmer's head, the APIs need to
    be complete, unsurprising, well documented, intuitive, unambiguous,
    predictable and reliable. As soon as people have to start digging into
    the source code to try and understand an API, the API has failed.
    In the interim, prose documentation that at least expresses what we
    believe
    to be true about this system will go a long way toward helping us
    understand what is true. *Then* maybe we can codify that in a spec as
    well.
    As many of you know, I wrote the first RabbitMQ and have written many of
    the large subsystems of RabbitMQ too - the paging layer and the
    active/active (async) HA layer too. These are not simple things. Rabbit
    is written in Erlang. Erlang has an optional type system which we do
    use. However, there is *no* excuse for not having good prose, for
    example, the long essays at:


    http://hg.rabbitmq.com/rabbitmq-server/file/68d851c51cc3/src/rabbit_variable_queue.erl#l32
    http://hg.rabbitmq.com/rabbitmq-server/file/68d851c51cc3/src/gm.erl#l19

    http://hg.rabbitmq.com/rabbitmq-server/file/68d851c51cc3/src/rabbit_mirror_queue_coordinator.erl#l50

    http://hg.rabbitmq.com/rabbitmq-server/file/68d851c51cc3/src/rabbit_msg_store.erl#l177

    All of those I originally wrote and they have been maintained. There is
    obviously a maintenance burden for having such essays: they *must* be
    kept up to date. But equally, there's a maintenance gain: the purpose of
    such prose is to get people who've not seen that area of code thinking
    in the right way very quickly - the scene has been set, so to speak, for
    the code they are about to encounter. For more experienced developers,
    the prose refreshes their memory when they come back to a module 6
    months later. It very much helps to write such prose as a form of
    finalised brain dump: to note down everything that was in your mind when
    you wrote the code and should be in the mind of all others who tread
    that path.

    Our larger subsystem APIs are often separated out and documented
    independently, for example

    http://hg.rabbitmq.com/rabbitmq-server/file/68d851c51cc3/src/rabbit_backing_queue.erl

    Needless to say, we also have large test suites and we rely on them for
    regression tests.

    Now I'm not a proponent of TDD or even pervasive unit testing. I value
    integration and end-to-end tests for regression spotting, and that's
    about it. I tend to feel you should be able to reason about code and
    reason about your changes to code, and that is worth much more than some
    mocks and some unit tests. To be able to do this requires more of an
    understanding of how components interact with each other and what the
    contracts and invariants are (and consequently, encourages you to
    decouple components, which is always beneficial). To some extent, I feel
    some people think TDD absolves devs of the responsibility of having that
    level of understanding. This is most certainly a falsehood - even with
    TDD and pervasive testing, without that level of understanding, the
    tests will test the wrong thing.

    Complex areas of code which you're not going to be able to test due to
    the difficulty of hitting reliably a certain combination of events can
    only be reasoned about and documented through comments. For example:

    http://hg.rabbitmq.com/rabbitmq-server/file/68d851c51cc3/src/rabbit_msg_store.erl#l551
    client_read3 is an internal function (ultimately called from the public
    function read) when one queue is trying to read a message off disk. But
    there could well be a GC (parallel and concurrent!) of the on-disk store
    involving the file containing the requested message at the same time.
    There are many possible combination of events regarding locking of that
    file, interaction with the GC, the possibility that the message has been
    moved by the GC but only after we first looked up its location, etc etc.
    You just can't test all these cases. So litter the code with comments
    that explain your thinking and hope that the peer code review either
    agrees, or better yet, manages to come up with some other combination of
    events that you've not thought of. If after five years of users using
    your code you never get a bug report in that area, then either the code
    is correct, or it's never used, or it falls in the
    acceptable-defect-rate bucket.

    All that said I may well be speaking from a position of ignorance about
    Ruby and TDD with Ruby, and even more so with Bosh - I've observed one
    day's worth of PP with the backend/services team and that's about it. So
    these comments should certainly be taken in a very general (and probably
    rather ill-informed) sense. My general fears about specs and mocks is
    that people only tend to test the common cases. They test what the
    feature in tracker says the feature should do. There's no negative
    testing, there's no probing of the interesting corner cases, and there
    seems to be little defensive coding in general. A large complex system
    has inevitably a large number of invariants and assumptions that a dev
    and a user needs to be aware of, and these things are seldom well
    captured by specs, IMO. Fify *words* of English which describe what you
    can and cannot do with an API are worth much more to me *as a user* (and
    probably also as a dev) than fifty *lines* of a spec that test one or
    two known good code paths, but don't actually cover what I want to use
    the API for.

    For Rabbit, and our test suites, we try to get as near to 100% code
    coverage as we can. Most modules we're over 90%. For some of the pure
    functional modules, 100% is possible. However, 100% code coverage means
    *bugger all* in the grand scheme of things. 100% code *path* coverage
    would be meaningful, but is impossible to achieve in most cases. For
    some areas, we have gone down the quick-check path. I would encourage
    people to use quick-check and similar tools: they are very useful, and
    go some distance further than just covering lines of code: if you have a
    well written quick-check soak test run a week or more on some code that
    is beyond mentally reasoning about, then you might have some degree of
    belief in the correctness of that code. If you're lucky. There are areas
    of code in Rabbit which are so complex that we have done normal tests,
    quick check tests, we've used the type system, and we've even written
    discrete event simulators to try and prove that the code copes with
    every possible combination of events. But give it two years of users
    banging on it, and bugs will eventually creep out. That's just the
    nature of trying to solve hard problems.

    Apologies if this is all unnecessarily preaching-to-the-converted, or
    just pointlessly ranty. Not my intention at all.

    Matthew
  • Abhi Hiremagalur at Jun 25, 2013 at 3:47 am

    Can we use bosh's own source as the basis of the style guide instead? It
    had a very consistent style. Someone else's styleguide could have been
    edited for the existing bosh style.
    With 3,500+ stars and a companion linter, the Ruby Style
    Guide<https://github.com/bbatsov/ruby-style-guide> seemed
    like a convenient & inexpensive way to apply *some* style consistently. I
    apologize for not noticing the consistency in the style of quotes that
    you've called out.

    Are there other existing styles you know about that might be documented in
    our .rubocop.yml files? Unfortunately, AFAIK, the quotes rule can only be
    turned off entirely not reversed.

    Re: "read well written specs" - next time I'm up in SF, I'll come visit and
    you can show me some well written specs that explain how bosh works. I've
    read them and I'm still confused how this beast of a program works :D
    It certainly can be a beast. I am optimistic we might gradually tame it :)

    To begin with it's been gratifying to watch our Code Climate score creep
    from 2.60 to 2.64 over the past few weeks. Hopefully we'll nudge that up
    even more in coming weeks.

    Massive nested Hashes being passed around. Written explanations of each
    Hash being passed in and out would make it much easier to comprehend and
    contribute, I think.
    I wonder if some of these hashes might be replaced with named
    classes<http://c2.com/cgi/wiki?PrimitiveObsession>,
    each with a spec describing its purpose. I am pretty new to this codebase
    so perhaps this is naive optimism on my part. I hope it isn't.

    Bosh is a huge code base, with lots of moving parts that aren't easy to run
    locally together.
    Agreed. Let's make it smaller and easier to run locally.

    I think more documentation of public & internal aspects is a worthy goal
    to improve the rate of onboarding of new full-time and new part-time devs
    to get into the mindset of the problem that is being solved.
    In ramping up as a bosh developer over these past few weeks I'd love to
    have had faster, more descriptive & reliable specs. I do think the folks
    who've come before me have been making these better recently, I'll try to
    help continue this.

    I am not confident that specs communicate as efficiently as the written
    word.
    I've had a consistently pleasant experience communicating through
    expressive specs on past projects, I'm optimistic.

    I think we similar goals for this codebase - to make it easier to reason
    about and consequently make BOSH easier to improve.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupbosh-dev @
postedJun 24, '13 at 7:40p
activeJun 25, '13 at 2:06p
posts13
users6

People

Translate

site design / logo © 2021 Grokbase