On 2 July 2013 18:43, Noah Misch wrote:
On Tue, Jul 02, 2013 at 10:17:08AM -0400, Robert Haas wrote:
So I think the first question we need to answer is: Should we just
reject Robins' patches en masse? If we do that, then the rest of this
is moot. If we don't do that, then the second question is whether we
should try to introduce a new schedule, and if so, whether we should
split out that new schedule before or after committing these patches
as they stand.
It's sad to simply reject meaningful automated tests on the basis of doubt
that they're important enough to belong in every human-in-the-loop test
I share the broader vision for automated testing represented by these

+1 We should be encouraging people to submit automated tests.

I share the annoyance of increasing the length of the automated test runs,
and have watched them get longer and longer over time. But I run the tests
because I want to see whether I broke anything and I stopped sitting and
watching them run long ago.

Automated testing is about x10-100 faster than manual testing, so I see new
tests as saving me time not wasting it.

Here are my opinions, for what they are worth. First, I think that
rejecting these new tests is a bad idea, although I've looked them
over a bit and I think there might be individual things we might want
to take out. Second, I think that creating a new schedule is likely
to cost developers more time than it saves them. Sure, you'll be able
to run the tests slightly faster, but when you commit something, break
the buildfarm (which does run those tests), and then have to go back
and fix the tests (or your patch), you'll waste more time doing that
than you saved by avoiding those few extra seconds of runtime. Third,
I think if we do want to create a new schedule, it makes more sense to
commit the tests first and then split out the new schedule according
to some criteria that we devise. There should be a principled reason
for putting tests in one schedule or the other; "all the tests that
Robins Tharakan wrote" is not a good filter criteria.
+1 for that plan. I don't know whether placing certain tests outside the
test sequence would indeed cost more time than it saves. I definitely
that if these new tests should appear elsewhere, some of our existing tests
should also move there.
Let's have a new schedule called minute-check with the objective to run the
common tests in 60 secs.

We can continue to expand the normal schedules from here.

Anybody that wants short tests can run that, everyone else can run the full
test suite.

We should be encouraging people to run the full test suite, not the fast

  Simon Riggs http://www.2ndQuadrant.com/
  PostgreSQL Development, 24x7 Support, Training & Services

Search Discussions

Discussion Posts


Follow ups

Related Discussions



site design / logo © 2021 Grokbase