FAQ
There are a number of changes we'd probably like to make to the way
things work in Postgres. This thread is not about discussing what
those are, just to say that requirements exist and have been discussed
in various threads over time.

The constraint on such changes is that we've decided that we must have
an upgrade path from release to release.

So I'd like to make a formal suggestion of a plan for how we cope with this:

1. Implement online upgrade in 9.4 via the various facilities we have
in-progress. That looks completely possible.

2. Name the next release after that 10.0 (would have been 9.5). We
declare now that
a) 10.0 will support on-line upgrade from 9.4 (only)
b) various major incompatibilities will be introduced in 10.0 - the
change in release number will indicate to everybody that is the case
c) agree that there will be no pg_upgrade patch from 9.4 to 10.0, so
that we will not be constrained by that

This plan doesn't presume any particular change. Each change would
need to be discussed on a separate thread, with a separate case for
each. All I'm suggesting is that we have a coherent plan for the
timing of such changes, so we can bundle them together into one
release.

By doing this now we give ourselves lots of time to plan changes that
will see us good for another decade. If we don't do this, then we
simply risk losing the iniative by continuing to support legacy
formats and approaches.

--
  Simon Riggs http://www.2ndQuadrant.com/
  PostgreSQL Development, 24x7 Support, Training & Services

Search Discussions

  • Merlin Moncure at May 25, 2013 at 4:27 pm

    On Sat, May 25, 2013 at 4:39 AM, Simon Riggs wrote:
    There are a number of changes we'd probably like to make to the way
    things work in Postgres. This thread is not about discussing what
    those are, just to say that requirements exist and have been discussed
    in various threads over time.

    The constraint on such changes is that we've decided that we must have
    an upgrade path from release to release.

    So I'd like to make a formal suggestion of a plan for how we cope with this:

    1. Implement online upgrade in 9.4 via the various facilities we have
    in-progress. That looks completely possible.

    2. Name the next release after that 10.0 (would have been 9.5). We
    declare now that
    a) 10.0 will support on-line upgrade from 9.4 (only)
    b) various major incompatibilities will be introduced in 10.0 - the
    change in release number will indicate to everybody that is the case
    c) agree that there will be no pg_upgrade patch from 9.4 to 10.0, so
    that we will not be constrained by that

    This plan doesn't presume any particular change. Each change would
    need to be discussed on a separate thread, with a separate case for
    each. All I'm suggesting is that we have a coherent plan for the
    timing of such changes, so we can bundle them together into one
    release.

    By doing this now we give ourselves lots of time to plan changes that
    will see us good for another decade. If we don't do this, then we
    simply risk losing the iniative by continuing to support legacy
    formats and approaches.
    Huh. I don't think that bumping the version number to 10.0 vs 9.5 is
    justification to introduce breaking changes. In fact, I would rather
    see 10.0 be the version where we formally stop doing that. I
    understand that some stuff needs to be improved but it often doesn't
    seem to be worth the cost in the long run.

    merlin
  • Merlin Moncure at May 28, 2013 at 3:04 pm

    On Sat, May 25, 2013 at 11:27 AM, Merlin Moncure wrote:
    On Sat, May 25, 2013 at 4:39 AM, Simon Riggs wrote:
    There are a number of changes we'd probably like to make to the way
    things work in Postgres. This thread is not about discussing what
    those are, just to say that requirements exist and have been discussed
    in various threads over time.

    The constraint on such changes is that we've decided that we must have
    an upgrade path from release to release.

    So I'd like to make a formal suggestion of a plan for how we cope with this:

    1. Implement online upgrade in 9.4 via the various facilities we have
    in-progress. That looks completely possible.

    2. Name the next release after that 10.0 (would have been 9.5). We
    declare now that
    a) 10.0 will support on-line upgrade from 9.4 (only)
    b) various major incompatibilities will be introduced in 10.0 - the
    change in release number will indicate to everybody that is the case
    c) agree that there will be no pg_upgrade patch from 9.4 to 10.0, so
    that we will not be constrained by that

    This plan doesn't presume any particular change. Each change would
    need to be discussed on a separate thread, with a separate case for
    each. All I'm suggesting is that we have a coherent plan for the
    timing of such changes, so we can bundle them together into one
    release.

    By doing this now we give ourselves lots of time to plan changes that
    will see us good for another decade. If we don't do this, then we
    simply risk losing the iniative by continuing to support legacy
    formats and approaches.
    Huh. I don't think that bumping the version number to 10.0 vs 9.5 is
    justification to introduce breaking changes. In fact, I would rather
    see 10.0 be the version where we formally stop doing that. I
    understand that some stuff needs to be improved but it often doesn't
    seem to be worth the cost in the long run.
    Please disregard this comment -- I didn't realize the topic was
    regarding on disk format -- I mistakenly though it was opening the
    door for user level feature changes.

    merlin
  • Jeff Davis at May 25, 2013 at 5:13 pm

    On Sat, 2013-05-25 at 10:39 +0100, Simon Riggs wrote:
    The constraint on such changes is that we've decided that we must have
    an upgrade path from release to release.
    Is this proposal only relaxing the binary upgrade requirement, or would
    it also relax other compatibility requirements, such as language and API
    compatibility?

    We need a couple major drivers of the incompatibility that really show
    users some value for going through the upgrade pain. Preferably, at
    least one would be a serious performance boost, because the users that
    encounter the most logical upgrade pain are also the ones that need a
    performance boost the most.

    Before we set a specific schedule, I think it would be a good idea to
    start prototyping some performance improvements that involve breaking
    the data format. Then, depending on how achievable it is, we can plan
    for however many more 9.X releases we think we need. That being said, I
    agree with you that planning in advance is important here, so that
    everyone knows when they need to get format-breaking changes in by.

    Regards,
      Jeff Davis
  • Simon Riggs at May 25, 2013 at 5:57 pm

    On 25 May 2013 18:13, Jeff Davis wrote:
    On Sat, 2013-05-25 at 10:39 +0100, Simon Riggs wrote:
    The constraint on such changes is that we've decided that we must have
    an upgrade path from release to release.
    Is this proposal only relaxing the binary upgrade requirement, or would
    it also relax other compatibility requirements, such as language and API
    compatibility?
    I'm suggesting that as many as possible changes we would like to make
    can happen in one release. This is for the benefit of users, so we
    dont make every release a source of incompatibilities.

    And that release should be the first one where we have online upgrade
    possible, which is one after next.

    --
      Simon Riggs http://www.2ndQuadrant.com/
      PostgreSQL Development, 24x7 Support, Training & Services
  • Bruce Momjian at May 25, 2013 at 8:44 pm

    On Sat, May 25, 2013 at 10:39:30AM +0100, Simon Riggs wrote:
    There are a number of changes we'd probably like to make to the way
    things work in Postgres. This thread is not about discussing what
    those are, just to say that requirements exist and have been discussed
    in various threads over time.

    The constraint on such changes is that we've decided that we must have
    an upgrade path from release to release.

    So I'd like to make a formal suggestion of a plan for how we cope with this:

    1. Implement online upgrade in 9.4 via the various facilities we have
    in-progress. That looks completely possible.

    2. Name the next release after that 10.0 (would have been 9.5). We
    declare now that
    a) 10.0 will support on-line upgrade from 9.4 (only)
    b) various major incompatibilities will be introduced in 10.0 - the
    change in release number will indicate to everybody that is the case
    c) agree that there will be no pg_upgrade patch from 9.4 to 10.0, so
    that we will not be constrained by that
    Assuming online upgrade is going to require logical replication, you are
    also assuming 2x storage as you need to have a second cluster to perform
    the upgrade. pg_upgrade would still be needed to upgrade a cluster
    in-place.

    This sounds like, "I created a new tool which does some of what the old
    tool does. Let's break the old tool to allow some unspecified changes I
    might want to make." I consider this thread to be not thought-through,
    obviously.

    --
       Bruce Momjian <bruce@momjian.us> http://momjian.us
       EnterpriseDB http://enterprisedb.com

       + It's impossible for everything to be true. +
  • Simon Riggs at May 26, 2013 at 9:53 am

    On 25 May 2013 21:44, Bruce Momjian wrote:
    On Sat, May 25, 2013 at 10:39:30AM +0100, Simon Riggs wrote:
    There are a number of changes we'd probably like to make to the way
    things work in Postgres. This thread is not about discussing what
    those are, just to say that requirements exist and have been discussed
    in various threads over time.

    The constraint on such changes is that we've decided that we must have
    an upgrade path from release to release.

    So I'd like to make a formal suggestion of a plan for how we cope with this:

    1. Implement online upgrade in 9.4 via the various facilities we have
    in-progress. That looks completely possible.

    2. Name the next release after that 10.0 (would have been 9.5). We
    declare now that
    a) 10.0 will support on-line upgrade from 9.4 (only)
    b) various major incompatibilities will be introduced in 10.0 - the
    change in release number will indicate to everybody that is the case
    c) agree that there will be no pg_upgrade patch from 9.4 to 10.0, so
    that we will not be constrained by that
    Assuming online upgrade is going to require logical replication, you are
    also assuming 2x storage as you need to have a second cluster to perform
    the upgrade.
    The people that want online upgrade already have 1+ other systems to
    do this with.

    pg_upgrade would still be needed to upgrade a cluster
    in-place.
    This sounds like, "I created a new tool which does some of what the old
    tool does. Let's break the old tool to allow some unspecified changes I
    might want to make."
    I haven't argued against pg_upgrade in general, nor said anything
    about breaking it. I proposed that we don't support a pg_upgrade path
    between two near-future releases, as a way of introducing
    incompatibilities. After that, we would continue to use pg_upgrade for
    later releases.

    Logical replication is being developed, which gives us a complete code
    path for doing what we'd need to do. The most important thing is we
    wouldn't need to develop any other code that exists just for upgrade.

    Writing special code just for pg_upgrade will take a lot of work.
    Running that code would mean pg_upgrade would touch the actual
    database, which would be down for a long time while it runs. And if it
    hits a bug during or after, you're hosed. So you'd need to take a full
    backup before you started the process, probably storing it on disk
    somewhere and so you would need x2 disk space with this route also.
    Specialised code is less well tested, which means bugs are more likely
    to occur and tends to perform more poorly. Not only that, but the
    first person to want an incompatibility gets to write all the code
    needed and take responsibility for the bugs. I can't comment for
    others, but I can say I would not personally choose that route - it
    looks both expensive and risky.

    I consider this thread to be not thought-through, obviously.
    My proposal has had lots of serious consideration, but that is not the
    topic of this thread.

    The title of the thread is a general one, with a clear objective.

    I'm looking for a way forwards that allows us to introduce the changes
    that many have proposed and which regrettably result in
    incompatibilities. If we have no plan I think its likely it will never
    happen and it is currently blocking useful change.

    Please explain what you consider to be a better plan, so we can judge
    all proposals together.

    --
      Simon Riggs http://www.2ndQuadrant.com/
      PostgreSQL Development, 24x7 Support, Training & Services
  • Bruce Momjian at May 26, 2013 at 1:18 pm

    On Sun, May 26, 2013 at 10:53:37AM +0100, Simon Riggs wrote:
    I consider this thread to be not thought-through, obviously.
    My proposal has had lots of serious consideration, but that is not the
    topic of this thread.

    The title of the thread is a general one, with a clear objective.

    I'm looking for a way forwards that allows us to introduce the changes
    that many have proposed and which regrettably result in
    incompatibilities. If we have no plan I think its likely it will never
    happen and it is currently blocking useful change.

    Please explain what you consider to be a better plan, so we can judge
    all proposals together.
    I agree with the idea of using logical replication as a way to do
    pg_upgrade version-breaking releases. What I don't know is what
    incompatible changes are pending that would require this.

    --
       Bruce Momjian <bruce@momjian.us> http://momjian.us
       EnterpriseDB http://enterprisedb.com

       + It's impossible for everything to be true. +
  • Bruce Momjian at May 26, 2013 at 1:22 pm

    On Sun, May 26, 2013 at 09:18:11AM -0400, Bruce Momjian wrote:
    On Sun, May 26, 2013 at 10:53:37AM +0100, Simon Riggs wrote:
    I consider this thread to be not thought-through, obviously.
    My proposal has had lots of serious consideration, but that is not the
    topic of this thread.

    The title of the thread is a general one, with a clear objective.

    I'm looking for a way forwards that allows us to introduce the changes
    that many have proposed and which regrettably result in
    incompatibilities. If we have no plan I think its likely it will never
    happen and it is currently blocking useful change.

    Please explain what you consider to be a better plan, so we can judge
    all proposals together.
    I agree with the idea of using logical replication as a way to do
    pg_upgrade version-breaking releases. What I don't know is what
    incompatible changes are pending that would require this.
    Sorry I was unclear. When I said "not thought-through", I meant, you
    need to start with the _reason_ we need to break pg_upgrade in an
    upcoming version, then we can start to plan how to do it. The logical
    replication idea is a good one for getting us through pg_upgrade
    version-breaking releases.

    I am fine with breaking pg_upgrade, but I just don't see the pending
    reason at this point.

    --
       Bruce Momjian <bruce@momjian.us> http://momjian.us
       EnterpriseDB http://enterprisedb.com

       + It's impossible for everything to be true. +
  • Hannu Krosing at May 26, 2013 at 2:26 pm

    On 05/26/2013 04:22 PM, Bruce Momjian wrote:
    On Sun, May 26, 2013 at 09:18:11AM -0400, Bruce Momjian wrote:
    On Sun, May 26, 2013 at 10:53:37AM +0100, Simon Riggs wrote:
    I consider this thread to be not thought-through, obviously.
    My proposal has had lots of serious consideration, but that is not the
    topic of this thread.

    The title of the thread is a general one, with a clear objective.

    I'm looking for a way forwards that allows us to introduce the changes
    that many have proposed and which regrettably result in
    incompatibilities. If we have no plan I think its likely it will never
    happen and it is currently blocking useful change.

    Please explain what you consider to be a better plan, so we can judge
    all proposals together.
    I agree with the idea of using logical replication as a way to do
    pg_upgrade version-breaking releases. What I don't know is what
    incompatible changes are pending that would require this.
    Sorry I was unclear. When I said "not thought-through", I meant, you
    need to start with the _reason_ we need to break pg_upgrade in an
    upcoming version, then we can start to plan how to do it. The logical
    replication idea is a good one for getting us through pg_upgrade
    version-breaking releases.

    I am fine with breaking pg_upgrade, but I just don't see the pending
    reason at this point.
    Not sure which ones Simon meant, but at least any new/better
    storage manager would seem to me to be requiring
    a non-pg_upgrade upgrade path unless we require the storage manager
    to also include a parallel implementation of pg_upgrade.

    The family of possible storage magers here would include column stores,
    distributed / partitioned / replicated memory-only / index-structured / ...
    storages which all could have advantages in certain situations and whic
    all
    need an upgrade path.

    While you could do this using sequance of first pg_upgrading and then doing
    some internal data migration to new storage manager doing this in one go
    would be much smoother.

    --
    Hannu Krosing
    PostgreSQL Consultant
    Performance, Scalability and High Availability
    2ndQuadrant Nordic OÜ
  • Josh Berkus at May 26, 2013 at 3:18 pm

    Not sure which ones Simon meant, but at least any new/better
    storage manager would seem to me to be requiring
    a non-pg_upgrade upgrade path unless we require the storage manager
    to also include a parallel implementation of pg_upgrade.
    Isn't this a bit of horse-cart inversion here? We just hashed out a
    tentative, incomplete pseudo-spec for storage managers *yesterday*. We
    don't have a complete spec at this point, let alone a development plan,
    and it's entirely possible that we'll be able to implement SMs without
    breaking pgupgrade.

    It's also not at all clear that we can develop SMs in less than 2 years.
      I tend to think it unlikely.

    First, let's have a few features for which breaking binary compatibility
    is a necessity or a clear benefit. Then we'll schedule when to break them.

    --
    Josh Berkus
    PostgreSQL Experts Inc.
    http://pgexperts.com
  • Stephen Frost at May 27, 2013 at 1:18 am

    * Josh Berkus (josh@agliodbs.com) wrote:
    and it's entirely possible that we'll be able to implement SMs without
    breaking pgupgrade.
    I'd certainly hope so.. It's certainly not obvious, to me at least,
    why a new SM or supporting any of those features would require
    breaking pg_upgrade. Perhaps there's something I'm not seeing there,
    but it had better be a *really* good reason..

    btw, has anyone posted the SM API proposal..? Unfortunately, I think I
    had to leave before that was hashed out..
    First, let's have a few features for which breaking binary compatibility
    is a necessity or a clear benefit. Then we'll schedule when to break them.
    Agreed.

      Thanks,

       Stephen
  • Tom Lane at May 27, 2013 at 1:49 am

    Stephen Frost writes:
    btw, has anyone posted the SM API proposal..? Unfortunately, I think I
    had to leave before that was hashed out..
    There isn't one yet. We think we understand where the pain points are,
    but there's still a long way to go to have a proposal.

        regards, tom lane
  • Christopher Browne at May 27, 2013 at 3:14 am
    The assumption that we ought to plan expressly for an incompatibility that
    essentially discards pg_upgrade seems premature, particularly in advance of
    would-be solutions that, in some cases, mightn't actually work.

    If pg_upgrade doesn't work, then, at present, the plausible solutions are
    to either dump and restore, which might take way too long, or use one of
    the logical replication systems (e.g. - Slony, Londiste, or similar, in the
    absence of the would-be built-in logical replication).

    Unfortunately, there are significant scenarios where none of these work,
    particularly for data warehouse-like systems where the database size is so
    large that the users cannot afford the disk space to construct a replica.
    It sure seems premature to intentionally leave that set of users out in the
    cold.
  • Bruce Momjian at May 27, 2013 at 12:07 pm

    On Sun, May 26, 2013 at 09:18:41PM -0400, Stephen Frost wrote:
    * Josh Berkus (josh@agliodbs.com) wrote:
    and it's entirely possible that we'll be able to implement SMs without
    breaking pgupgrade.
    I'd certainly hope so.. It's certainly not obvious, to me at least,
    why a new SM or supporting any of those features would require
    breaking pg_upgrade. Perhaps there's something I'm not seeing there,
    but it had better be a *really* good reason..
    If I had to _guess_, I would say users who are using the default storage
    manager would still be able to use pg_upgrade, and those using
    non-default storage managers perhaps can't.

    But, again, this is all so hypothetical that it doesn't seem worth
    talking about. My big point is that someone came to me at PGCon asking
    if I knew anything about why Simon thought we needed to break pg_upgrade
    in <2 years, and I said no, so I had go digging into my email to find
    out what was going on. Simon has a very visible position in the
    community, so when he suggests something, people take it seriously,
    which means I have to address it. I would prefer if there was more
    thought put into the ideas before they are posted.

    --
       Bruce Momjian <bruce@momjian.us> http://momjian.us
       EnterpriseDB http://enterprisedb.com

       + It's impossible for everything to be true. +
  • Stephen Frost at May 27, 2013 at 12:27 pm

    * Bruce Momjian (bruce@momjian.us) wrote:
    If I had to _guess_, I would say users who are using the default storage
    manager would still be able to use pg_upgrade, and those using
    non-default storage managers perhaps can't.
    That would make sense.
    But, again, this is all so hypothetical that it doesn't seem worth
    talking about.
    Having a specific list of "these are the things we want to change, and
    why, and here is why pg_upgrade can't support it" would be much more
    useful to work from, I agree.

    That said, many discussions and ideas do get shut down, perhaps too
    early, because of pg_upgrade considerations. If we had a plan to have
    an incompatible release in the future, those ideas and discussions might
    be able to progress to a point where we determine it's worth it to take
    the pain of a non-pg_upgrade-supported release. That's a bit of a
    stretch, in my view, but I suppose it's possible. Even so though, I
    would suggest that we put together a wiki page to list out those items
    and encourage people to add to such a list; perhaps having an item on
    that list would make discussion about it progress beyond "it breaks
    pg_upgrade".

      Thanks,

       Stephen
  • Bruce Momjian at May 27, 2013 at 1:18 pm

    On Mon, May 27, 2013 at 08:26:48AM -0400, Stephen Frost wrote:
    * Bruce Momjian (bruce@momjian.us) wrote:
    If I had to _guess_, I would say users who are using the default storage
    manager would still be able to use pg_upgrade, and those using
    non-default storage managers perhaps can't.
    That would make sense.
    But, again, this is all so hypothetical that it doesn't seem worth
    talking about.
    Having a specific list of "these are the things we want to change, and
    why, and here is why pg_upgrade can't support it" would be much more
    useful to work from, I agree.

    That said, many discussions and ideas do get shut down, perhaps too
    early, because of pg_upgrade considerations. If we had a plan to have
    an incompatible release in the future, those ideas and discussions might
    be able to progress to a point where we determine it's worth it to take
    the pain of a non-pg_upgrade-supported release. That's a bit of a
    stretch, in my view, but I suppose it's possible. Even so though, I
    would suggest that we put together a wiki page to list out those items
    and encourage people to add to such a list; perhaps having an item on
    that list would make discussion about it progress beyond "it breaks
    pg_upgrade".
    Yes, we should be collecting things we want to do for a pg_upgrade break
    so we can see the list all in one place.

    --
       Bruce Momjian <bruce@momjian.us> http://momjian.us
       EnterpriseDB http://enterprisedb.com

       + It's impossible for everything to be true. +
  • Tom Lane at May 27, 2013 at 2:36 pm

    Bruce Momjian writes:
    On Mon, May 27, 2013 at 08:26:48AM -0400, Stephen Frost wrote:
    That said, many discussions and ideas do get shut down, perhaps too
    early, because of pg_upgrade considerations. If we had a plan to have
    an incompatible release in the future, those ideas and discussions might
    be able to progress to a point where we determine it's worth it to take
    the pain of a non-pg_upgrade-supported release. That's a bit of a
    stretch, in my view, but I suppose it's possible. Even so though, I
    would suggest that we put together a wiki page to list out those items
    and encourage people to add to such a list; perhaps having an item on
    that list would make discussion about it progress beyond "it breaks
    pg_upgrade".
    Yes, we should be collecting things we want to do for a pg_upgrade break
    so we can see the list all in one place.
    Precisely. We've said right along that we reserve the right to have a
    non-upgradable disk format change whenever sufficiently many reasons
    accumulate to do that. The way to go about that is to collect projects
    that need to be kept on hold for such a release --- not to say we're
    going to have such a release and then look for reasons.

        regards, tom lane
  • Alvaro Herrera at May 27, 2013 at 3:38 pm

    Tom Lane wrote:
    Bruce Momjian <bruce@momjian.us> writes:
    Yes, we should be collecting things we want to do for a pg_upgrade break
    so we can see the list all in one place.
    Precisely. We've said right along that we reserve the right to have a
    non-upgradable disk format change whenever sufficiently many reasons
    accumulate to do that.
    Do we have a wiki page about this?

    --
    Álvaro Herrera http://www.2ndQuadrant.com/
    PostgreSQL Development, 24x7 Support, Training & Services
  • Simon Riggs at May 27, 2013 at 4:42 pm

    On 27 May 2013 15:36, Tom Lane wrote:
    Bruce Momjian <bruce@momjian.us> writes:
    On Mon, May 27, 2013 at 08:26:48AM -0400, Stephen Frost wrote:
    That said, many discussions and ideas do get shut down, perhaps too
    early, because of pg_upgrade considerations. If we had a plan to have
    an incompatible release in the future, those ideas and discussions might
    be able to progress to a point where we determine it's worth it to take
    the pain of a non-pg_upgrade-supported release. That's a bit of a
    stretch, in my view, but I suppose it's possible. Even so though, I
    would suggest that we put together a wiki page to list out those items
    and encourage people to add to such a list; perhaps having an item on
    that list would make discussion about it progress beyond "it breaks
    pg_upgrade".
    Yes, we should be collecting things we want to do for a pg_upgrade break
    so we can see the list all in one place.
    Precisely. We've said right along that we reserve the right to have a
    non-upgradable disk format change whenever sufficiently many reasons
    accumulate to do that.
    I'm happy with that.

    I was also thinking about collecting changes not related just to disk
    format, if any exist.
    The way to go about that is to collect projects
    that need to be kept on hold for such a release --- not to say we're
    going to have such a release and then look for reasons.
    Agreed.

    I was trying to establish a realistic timeline for such events, so
    that the planning was able to be taken seriously. Yes, it wass a "work
    backwards" or "what if" type of planning. But now we have a rough plan
    of how it might look, collecting ideas can begin.

    --
      Simon Riggs http://www.2ndQuadrant.com/
      PostgreSQL Development, 24x7 Support, Training & Services
  • Craig Ringer at May 27, 2013 at 11:58 pm

    On 05/28/2013 12:41 AM, Simon Riggs wrote:
    I'm happy with that.

    I was also thinking about collecting changes not related just to disk
    format, if any exist.
    Any wire protocol or syntax changes?

    I can't seem to find a "things we want to do in wire protocol v4" doc in
    the wiki but I know I've seen occasional discussion of things that can't
    be done without protocol changes. Anyone with a better memory than me
    able to pitch in?

    What'd be required to support in-band query cancellation? Sending
    per-statement GUCs (to allow true statement timeout)?

    I can't think of any major syntax warts and grandfathered quirks that'd
    be really great to get rid of if we had the freedom to break things.

    --
      Craig Ringer http://www.2ndQuadrant.com/
      PostgreSQL Development, 24x7 Support, Training & Services
  • Joshua D. Drake at May 28, 2013 at 12:21 am

    On 05/27/2013 04:58 PM, Craig Ringer wrote:
    On 05/28/2013 12:41 AM, Simon Riggs wrote:
    I'm happy with that.

    I was also thinking about collecting changes not related just to disk
    format, if any exist.
    Any wire protocol or syntax changes?

    I can't seem to find a "things we want to do in wire protocol v4" doc in
    the wiki but I know I've seen occasional discussion of things that can't
    be done without protocol changes. Anyone with a better memory than me
    able to pitch in?

    What'd be required to support in-band query cancellation? Sending
    per-statement GUCs (to allow true statement timeout)?
    I would like to see the ability to define if a query is read only at the
    protocol level, so that load balances that speak libpq can know what to
    do with the query without parsing it.

    JD

    --
    Command Prompt, Inc. - http://www.commandprompt.com/ 509-416-6579
    PostgreSQL Support, Training, Professional Services and Development
    High Availability, Oracle Conversion, Postgres-XC, @cmdpromptinc
    For my dreams of your image that blossoms
         a rose in the deeps of my heart. - W.B. Yeats
  • Bruce Momjian at May 28, 2013 at 9:18 pm

    On Mon, May 27, 2013 at 05:21:16PM -0700, Joshua D. Drake wrote:
    On 05/27/2013 04:58 PM, Craig Ringer wrote:
    On 05/28/2013 12:41 AM, Simon Riggs wrote:
    I'm happy with that.

    I was also thinking about collecting changes not related just to disk
    format, if any exist.
    Any wire protocol or syntax changes?

    I can't seem to find a "things we want to do in wire protocol v4" doc in
    the wiki but I know I've seen occasional discussion of things that can't
    be done without protocol changes. Anyone with a better memory than me
    able to pitch in?

    What'd be required to support in-band query cancellation? Sending
    per-statement GUCs (to allow true statement timeout)?
    I would like to see the ability to define if a query is read only at
    the protocol level, so that load balances that speak libpq can know
    what to do with the query without parsing it.
    Sounds nice, but how would we do that? That would require libpq to know
    it, right? Do we pass anything back after parsing but before execution?
      Could it be optional? What about functions that modify the database
    --- isn't that only known at execution time?

    --
       Bruce Momjian <bruce@momjian.us> http://momjian.us
       EnterpriseDB http://enterprisedb.com

       + It's impossible for everything to be true. +
  • Joshua D. Drake at May 28, 2013 at 9:26 pm

    On 05/28/2013 02:18 PM, Bruce Momjian wrote:

    I would like to see the ability to define if a query is read only at
    the protocol level, so that load balances that speak libpq can know
    what to do with the query without parsing it.
    Sounds nice, but how would we do that? That would require libpq to know
    it, right? Do we pass anything back after parsing but before execution?
    Could it be optional? What about functions that modify the database
    --- isn't that only known at execution time?
    I can't speak to the actual C code that would be required but from a
    user space, I could see something like this:

    con = psycopg2.connect(database='testdb', user='test',
    transaction-type='r')

    Thus when the connection is made, before anything else is done, we know
    it is a read only connection and therefore any load balancer speaking
    libpq would also know it is a read only. The default of course would be
    r/w and you would use a different connection handler for r/w or w queries.

    The other option would be to do it on query execute but that doesn't
    seem as efficient as it would have to be parsed each time. Although it
    would still be better than reading the actual SQL.

    Sincerely,

    Joshua D. Drake




    --
    Command Prompt, Inc. - http://www.commandprompt.com/ 509-416-6579
    PostgreSQL Support, Training, Professional Services and Development
    High Availability, Oracle Conversion, Postgres-XC, @cmdpromptinc
    For my dreams of your image that blossoms
         a rose in the deeps of my heart. - W.B. Yeats
  • Bruce Momjian at May 28, 2013 at 10:37 pm

    On Tue, May 28, 2013 at 02:26:06PM -0700, Joshua D. Drake wrote:
    Sounds nice, but how would we do that? That would require libpq to know
    it, right? Do we pass anything back after parsing but before execution?
    Could it be optional? What about functions that modify the database
    --- isn't that only known at execution time?
    I can't speak to the actual C code that would be required but from a
    user space, I could see something like this:

    con = psycopg2.connect(database='testdb', user='test',
    transaction-type='r')

    Thus when the connection is made, before anything else is done, we
    know it is a read only connection and therefore any load balancer
    speaking libpq would also know it is a read only. The default of
    course would be r/w and you would use a different connection handler
    for r/w or w queries.

    The other option would be to do it on query execute but that doesn't
    seem as efficient as it would have to be parsed each time. Although
    it would still be better than reading the actual SQL.
    Well, you could do SET TRANSACTION READ ONLY, and that would prevent any
    write transactions. You could assume it is a read query, and get the
    error and resubmit on the master if that happens, but that sounds
    inefficient. I thought you were asking for something where you could
    submit a query and it would report back as read/write or read-only.

    --
       Bruce Momjian <bruce@momjian.us> http://momjian.us
       EnterpriseDB http://enterprisedb.com

       + It's impossible for everything to be true. +
  • Joshua D. Drake at May 28, 2013 at 10:39 pm

    On 05/28/2013 03:36 PM, Bruce Momjian wrote:

    The other option would be to do it on query execute but that doesn't
    seem as efficient as it would have to be parsed each time. Although
    it would still be better than reading the actual SQL.
    Well, you could do SET TRANSACTION READ ONLY, and that would prevent any
    write transactions. You could assume it is a read query, and get the
    error and resubmit on the master if that happens, but that sounds
    inefficient. I thought you were asking for something where you could
    submit a query and it would report back as read/write or read-only.
    No I am suggesting something that before anything happens with the
    parser, the protocol knows what is up. So things like pgpool-ii don't
    even need a parser, it just knows it is a read only query because the
    protocol says so.

    JD



    --
    Command Prompt, Inc. - http://www.commandprompt.com/ 509-416-6579
    PostgreSQL Support, Training, Professional Services and Development
    High Availability, Oracle Conversion, Postgres-XC, @cmdpromptinc
    For my dreams of your image that blossoms
         a rose in the deeps of my heart. - W.B. Yeats
  • Bruce Momjian at May 28, 2013 at 11:05 pm

    On Tue, May 28, 2013 at 03:39:10PM -0700, Joshua D. Drake wrote:
    On 05/28/2013 03:36 PM, Bruce Momjian wrote:

    The other option would be to do it on query execute but that doesn't
    seem as efficient as it would have to be parsed each time. Although
    it would still be better than reading the actual SQL.
    Well, you could do SET TRANSACTION READ ONLY, and that would prevent any
    write transactions. You could assume it is a read query, and get the
    error and resubmit on the master if that happens, but that sounds
    inefficient. I thought you were asking for something where you could
    submit a query and it would report back as read/write or read-only.
    No I am suggesting something that before anything happens with the
    parser, the protocol knows what is up. So things like pgpool-ii
    don't even need a parser, it just knows it is a read only query
    because the protocol says so.
    Oh, that is an interesting idea. The application is indicating it is
    read-only via the protocol, and poolers can optimize that. Don't we
    have the ability to pass arbitrary GUC values back through the protocol,
    e.g. transaction_read_only? If not, that might be a way to do this
    cleanly.

    --
       Bruce Momjian <bruce@momjian.us> http://momjian.us
       EnterpriseDB http://enterprisedb.com

       + It's impossible for everything to be true. +
  • Joshua D. Drake at May 28, 2013 at 11:37 pm

    On 05/28/2013 04:05 PM, Bruce Momjian wrote:
    On Tue, May 28, 2013 at 03:39:10PM -0700, Joshua D. Drake wrote:
    On 05/28/2013 03:36 PM, Bruce Momjian wrote:

    The other option would be to do it on query execute but that doesn't
    seem as efficient as it would have to be parsed each time. Although
    it would still be better than reading the actual SQL.
    Well, you could do SET TRANSACTION READ ONLY, and that would prevent any
    write transactions. You could assume it is a read query, and get the
    error and resubmit on the master if that happens, but that sounds
    inefficient. I thought you were asking for something where you could
    submit a query and it would report back as read/write or read-only.
    No I am suggesting something that before anything happens with the
    parser, the protocol knows what is up. So things like pgpool-ii
    don't even need a parser, it just knows it is a read only query
    because the protocol says so.
    Oh, that is an interesting idea. The application is indicating it is
    read-only via the protocol, and poolers can optimize that. Don't we
    have the ability to pass arbitrary GUC values back through the protocol,
    e.g. transaction_read_only? If not, that might be a way to do this
    cleanly.
    I don't know but I don't think so. Anything that is calling SET is going
    to run through the parser.

    JD

    --
    Command Prompt, Inc. - http://www.commandprompt.com/ 509-416-6579
    PostgreSQL Support, Training, Professional Services and Development
    High Availability, Oracle Conversion, Postgres-XC, @cmdpromptinc
    For my dreams of your image that blossoms
         a rose in the deeps of my heart. - W.B. Yeats
  • Tatsuo Ishii at May 29, 2013 at 1:02 am

    On 05/28/2013 04:05 PM, Bruce Momjian wrote:
    On Tue, May 28, 2013 at 03:39:10PM -0700, Joshua D. Drake wrote:
    On 05/28/2013 03:36 PM, Bruce Momjian wrote:

    The other option would be to do it on query execute but that doesn't
    seem as efficient as it would have to be parsed each time. Although
    it would still be better than reading the actual SQL.
    Well, you could do SET TRANSACTION READ ONLY, and that would prevent
    any
    write transactions. You could assume it is a read query, and get the
    error and resubmit on the master if that happens, but that sounds
    inefficient. I thought you were asking for something where you could
    submit a query and it would report back as read/write or read-only.
    No I am suggesting something that before anything happens with the
    parser, the protocol knows what is up. So things like pgpool-ii
    don't even need a parser, it just knows it is a read only query
    because the protocol says so.
    Oh, that is an interesting idea. The application is indicating it is
    read-only via the protocol, and poolers can optimize that. Don't we
    have the ability to pass arbitrary GUC values back through the
    protocol,
    e.g. transaction_read_only? If not, that might be a way to do this
    cleanly.
    I don't know but I don't think so. Anything that is calling SET is
    going to run through the parser.
    Right. SET command needs to be parsed by the parser. However, we
    already have embedded parameters in the start up packet, which needs
    to be recognized by pooler anyway. See "StartupMessage" section in:
    http://www.postgresql.org/docs/9.3/static/protocol-message-formats.html

    I am not sure backend currently permits to have
    default_transaction_read_only = on in the startup packet or not
    though.
    --
    Tatsuo Ishii
    SRA OSS, Inc. Japan
    English: http://www.sraoss.co.jp/index_en.php
    Japanese: http://www.sraoss.co.jp
  • Jaime Casanova at May 29, 2013 at 12:50 am

    On Tue, May 28, 2013 at 4:26 PM, Joshua D. Drake wrote:
    On 05/28/2013 02:18 PM, Bruce Momjian wrote:

    I would like to see the ability to define if a query is read only at
    the protocol level, so that load balances that speak libpq can know
    what to do with the query without parsing it.

    Sounds nice, but how would we do that? That would require libpq to know
    it, right? Do we pass anything back after parsing but before execution?
    Could it be optional? What about functions that modify the database
    --- isn't that only known at execution time?

    I can't speak to the actual C code that would be required but from a user
    space, I could see something like this:

    con = psycopg2.connect(database='testdb', user='test', transaction-type='r')

    Thus when the connection is made, before anything else is done, we know it
    is a read only connection and therefore any load balancer speaking libpq
    would also know it is a read only. The default of course would be r/w and
    you would use a different connection handler for r/w or w queries.
    you can do that today already, kind-of

    create an entry in pgbouncer that connect to
    host=read-only.servers.dns and make read-only.servers.dns to point to
    more than 1 ip.
    then when the application wants to do load balancing, just connect to
    the entry that points to read-only.servers.dns and let the magic
    happens

    which would be great is this to happen transparently to the application
    The other option would be to do it on query execute but that doesn't seem as
    efficient as it would have to be parsed each time. Although it would still
    be better than reading the actual SQL.
    another idea, as someone else mentioned, and i think has been
    discussed bedore is a function that says if the query is r-o or not...
    maybe even exporting the plan so we don't need to replan again...

    Not sure if that is possible, just hand waving...


    --
    Jaime Casanova www.2ndQuadrant.com
    Professional PostgreSQL: Soporte 24x7 y capacitación
    Phone: +593 4 5107566 Cell: +593 987171157
  • Alvaro Herrera at May 28, 2013 at 9:55 pm

    Bruce Momjian wrote:
    On Mon, May 27, 2013 at 05:21:16PM -0700, Joshua D. Drake wrote:

    I would like to see the ability to define if a query is read only at
    the protocol level, so that load balances that speak libpq can know
    what to do with the query without parsing it.
    Sounds nice, but how would we do that? That would require libpq to know
    it, right? Do we pass anything back after parsing but before execution?
    Could it be optional? What about functions that modify the database
    --- isn't that only known at execution time?
    Well, if you hit anything that tries to acquire an Xid, and you're in a
    context that said only read-only was acceptable, just raise an error.

    In a similar vein, I vaguely recall we discussed (after some security
    vulnerability involving SQL injection) a mode where we only accept only
    one command per PQexec() call, i.e. reject execution of commands that
    contain multiple queries.

    --
    Álvaro Herrera http://www.2ndQuadrant.com/
    PostgreSQL Development, 24x7 Support, Training & Services
  • Bruce Momjian at May 28, 2013 at 9:11 pm

    On Tue, May 28, 2013 at 07:58:33AM +0800, Craig Ringer wrote:
    On 05/28/2013 12:41 AM, Simon Riggs wrote:
    I'm happy with that.

    I was also thinking about collecting changes not related just to disk
    format, if any exist.
    Any wire protocol or syntax changes?

    I can't seem to find a "things we want to do in wire protocol v4" doc in
    the wiki but I know I've seen occasional discussion of things that can't
    be done without protocol changes. Anyone with a better memory than me
    able to pitch in?
    Sure, it is on the TODO list:

      https://wiki.postgresql.org/wiki/Todo#.2Fcontrib.2Fpg_upgrade

    I can only get a link to pg_upgrade from there, so look two sections
    below that for "Wire Protocol Changes".

    --
       Bruce Momjian <bruce@momjian.us> http://momjian.us
       EnterpriseDB http://enterprisedb.com

       + It's impossible for everything to be true. +
  • Craig Ringer at May 29, 2013 at 12:32 am

    On 05/29/2013 05:11 AM, Bruce Momjian wrote:

    Sure, it is on the TODO list:

    https://wiki.postgresql.org/wiki/Todo#.2Fcontrib.2Fpg_upgrade

    I can only get a link to pg_upgrade from there, so look two sections
    below that for "Wire Protocol Changes".
    Thanks.

    The direct link is
    https://wiki.postgresql.org/wiki/Todo#Wire_Protocol_Changes for anyone
    looking for it later.

    --
      Craig Ringer http://www.2ndQuadrant.com/
      PostgreSQL Development, 24x7 Support, Training & Services
  • Daniel Farina at May 29, 2013 at 6:04 am

    On Mon, May 27, 2013 at 9:41 AM, Simon Riggs wrote:
    On 27 May 2013 15:36, Tom Lane wrote:
    Bruce Momjian <bruce@momjian.us> writes:
    On Mon, May 27, 2013 at 08:26:48AM -0400, Stephen Frost wrote:
    That said, many discussions and ideas do get shut down, perhaps too
    early, because of pg_upgrade considerations. If we had a plan to have
    an incompatible release in the future, those ideas and discussions might
    be able to progress to a point where we determine it's worth it to take
    the pain of a non-pg_upgrade-supported release. That's a bit of a
    stretch, in my view, but I suppose it's possible. Even so though, I
    would suggest that we put together a wiki page to list out those items
    and encourage people to add to such a list; perhaps having an item on
    that list would make discussion about it progress beyond "it breaks
    pg_upgrade".
    Yes, we should be collecting things we want to do for a pg_upgrade break
    so we can see the list all in one place.
    Precisely. We've said right along that we reserve the right to have a
    non-upgradable disk format change whenever sufficiently many reasons
    accumulate to do that.
    Here's one that's come up a few times: being able to tweak the
    out-of-line storage strategy, e.g. change the compression format used.
      I think some folks were lamenting the lack of a convenient byte in
    the right place for that one.
  • Bruce Momjian at May 27, 2013 at 4:06 pm

    On Mon, May 27, 2013 at 09:17:50AM -0400, Bruce Momjian wrote:
    That said, many discussions and ideas do get shut down, perhaps too
    early, because of pg_upgrade considerations. If we had a plan to have
    an incompatible release in the future, those ideas and discussions might
    be able to progress to a point where we determine it's worth it to take
    the pain of a non-pg_upgrade-supported release. That's a bit of a
    stretch, in my view, but I suppose it's possible. Even so though, I
    would suggest that we put together a wiki page to list out those items
    and encourage people to add to such a list; perhaps having an item on
    that list would make discussion about it progress beyond "it breaks
    pg_upgrade".
    Yes, we should be collecting things we want to do for a pg_upgrade break
    so we can see the list all in one place.
    OK, I have added a section to the TODO list for this:

      Desired changes that would prevent upgrades with pg_upgrade

          32-bit page checksums

    Are there any others?

    --
       Bruce Momjian <bruce@momjian.us> http://momjian.us
       EnterpriseDB http://enterprisedb.com

       + It's impossible for everything to be true. +
  • Tom Lane at May 27, 2013 at 6:09 pm

    Bruce Momjian writes:
    On Mon, May 27, 2013 at 09:17:50AM -0400, Bruce Momjian wrote:
    Yes, we should be collecting things we want to do for a pg_upgrade break
    so we can see the list all in one place.
    OK, I have added a section to the TODO list for this:
    Desired changes that would prevent upgrades with pg_upgrade
    32-bit page checksums
    Are there any others?
    GiST indexes really oughta have a metapage so there can be a version
    number in them.

    Also, if we are going to unify hstore and json, it'd be nice if we could
    change the existing binary representation of hstore (per discussions at
    Oleg and Teodor's talk --- this will be moot if we invent a new core
    type, but it'd be better not to have to).

    There are probably some other data-type-specific cleanups we could
    make, but I have to go get on an airplane so no time to think about it.

        regards, tom lane
  • Bruce Momjian at May 28, 2013 at 9:20 pm

    On Mon, May 27, 2013 at 02:09:05PM -0400, Tom Lane wrote:
    Bruce Momjian <bruce@momjian.us> writes:
    On Mon, May 27, 2013 at 09:17:50AM -0400, Bruce Momjian wrote:
    Yes, we should be collecting things we want to do for a pg_upgrade break
    so we can see the list all in one place.
    OK, I have added a section to the TODO list for this:
    Desired changes that would prevent upgrades with pg_upgrade
    32-bit page checksums
    Are there any others?
    GiST indexes really oughta have a metapage so there can be a version
    number in them.

    Also, if we are going to unify hstore and json, it'd be nice if we could
    change the existing binary representation of hstore (per discussions at
    Oleg and Teodor's talk --- this will be moot if we invent a new core
    type, but it'd be better not to have to).

    There are probably some other data-type-specific cleanups we could
    make, but I have to go get on an airplane so no time to think about it.
    OK, GiST and hstore added to TODO list.

    --
       Bruce Momjian <bruce@momjian.us> http://momjian.us
       EnterpriseDB http://enterprisedb.com

       + It's impossible for everything to be true. +
  • Alvaro Herrera at May 27, 2013 at 7:06 pm

    Bruce Momjian wrote:

    OK, I have added a section to the TODO list for this:

    Desired changes that would prevent upgrades with pg_upgrade

    32-bit page checksums

    Are there any others?
    I would have each data segment be self-identifying, i.e. have a magic
    number at the beginning of the file and the relation OID, some fork
    identification and the segment number somewhere -- probably the special
    space of the first page.

    --
    Álvaro Herrera http://www.2ndQuadrant.com/
    PostgreSQL Development, 24x7 Support, Training & Services
  • Bruce Momjian at May 28, 2013 at 10:38 pm

    On Mon, May 27, 2013 at 03:06:13PM -0400, Alvaro Herrera wrote:
    Bruce Momjian wrote:
    OK, I have added a section to the TODO list for this:

    Desired changes that would prevent upgrades with pg_upgrade

    32-bit page checksums

    Are there any others?
    I would have each data segment be self-identifying, i.e. have a magic
    number at the beginning of the file and the relation OID, some fork
    identification and the segment number somewhere -- probably the special
    space of the first page.
    Is this something we want on the TODO? I was not clear how to do with
    without making the first page format special or wasting space on all the
    other pages.

    --
       Bruce Momjian <bruce@momjian.us> http://momjian.us
       EnterpriseDB http://enterprisedb.com

       + It's impossible for everything to be true. +
  • Alvaro Herrera at May 29, 2013 at 3:50 am

    Bruce Momjian wrote:
    On Mon, May 27, 2013 at 03:06:13PM -0400, Alvaro Herrera wrote:
    Bruce Momjian wrote:
    I would have each data segment be self-identifying, i.e. have a magic
    number at the beginning of the file and the relation OID, some fork
    identification and the segment number somewhere -- probably the special
    space of the first page.
    Is this something we want on the TODO? I was not clear how to do with
    without making the first page format special or wasting space on all the
    other pages.
    I don't think the special space has to necessarily be identically sized
    in all the pages -- I admit I haven't looked closely, but the special
    size is part of the page header.

    --
    Álvaro Herrera http://www.2ndQuadrant.com/
    PostgreSQL Development, 24x7 Support, Training & Services
  • Hannu Krosing at May 27, 2013 at 1:25 pm

    On 05/26/2013 06:18 PM, Josh Berkus wrote:
    Not sure which ones Simon meant, but at least any new/better
    storage manager would seem to me to be requiring
    a non-pg_upgrade upgrade path unless we require the storage manager
    to also include a parallel implementation of pg_upgrade.
    Isn't this a bit of horse-cart inversion here? We just hashed out a
    tentative, incomplete pseudo-spec for storage managers *yesterday*.
    Many people have been *thinking* about pluggable storage /
    storage managers for much longer time.
    We
    don't have a complete spec at this point, let alone a development plan,
    I think we will have a development plan *before* complete spec
    anyway :)
    and it's entirely possible that we'll be able to implement SMs without
    breaking pgupgrade.
    My point was exactly to not spend majority of new storage manager
    discussion on "does it break pg_upgrade", "maybe we can find a way
    to do it without breaking pg_upgrade", etc...
    It's also not at all clear that we can develop SMs in less than 2 years.
    I tend to think it unlikely.
    I think the important part of Simons message was not "two years"
    First, let's have a few features for which breaking binary compatibility
    is a necessity or a clear benefit. Then we'll schedule when to break them.
    But rather than "it breaks pg_upgrade" not being a complete stopper for
    proposed useful features that might break it.

    --
    Hannu Krosing
    PostgreSQL Consultant
    Performance, Scalability and High Availability
    2ndQuadrant Nordic OÜ
  • Craig Ringer at May 27, 2013 at 5:01 am

    On 05/25/2013 05:39 PM, Simon Riggs wrote:
    2. Name the next release after that 10.0 (would have been 9.5). We
    declare now that
    a) 10.0 will support on-line upgrade from 9.4 (only)
    b) various major incompatibilities will be introduced in 10.0 - the
    change in release number will indicate to everybody that is the case
    c) agree that there will be no pg_upgrade patch from 9.4 to 10.0, so
    that we will not be constrained by that
    While we're talking about changing things, what about:

    - Switching to single-major-version release numbering. The number of
    people who say "PostgreSQL 9.x" is amazing; even *packagers* get this
    wrong and produce "postgresql-9" packages. Witness Amazon Linux's awful
    PostgreSQL packages for example. Going to PostgreSQL 10.0, 11.0, 12.0,
    etc with a typical major/minor scheme might be worth considering.

    - s/cluster/server/g . Just because "cluster" is historical usage
    doesn't make it any less confusing for users.

    *dives for asbestos fire suit*

    --
      Craig Ringer http://www.2ndQuadrant.com/
      PostgreSQL Development, 24x7 Support, Training & Services
  • Michael Paquier at May 27, 2013 at 9:45 am

    On Mon, May 27, 2013 at 2:01 PM, Craig Ringer wrote:

    On 05/25/2013 05:39 PM, Simon Riggs wrote:
    - Switching to single-major-version release numbering. The number of
    people who say "PostgreSQL 9.x" is amazing; even *packagers* get this
    wrong and produce "postgresql-9" packages. Witness Amazon Linux's awful
    PostgreSQL packages for example. Going to PostgreSQL 10.0, 11.0, 12.0,
    etc with a typical major/minor scheme might be worth considering.
    In this case you don't even need the 2nd digit...
    Btw, -1 for the idea, as it would remove the possibility to tell that a new
    major release incrementing the 1st digit of version number brings more
    enhancement than normal major releases incrementing the 1st digit. This was
    the case for 9.0, helping people in remembering that streaming replication
    has been introduced from 9.x series.
    --
    Michael
  • Alvaro Herrera at May 27, 2013 at 3:37 pm

    Michael Paquier escribió:
    On Mon, May 27, 2013 at 2:01 PM, Craig Ringer wrote:

    On 05/25/2013 05:39 PM, Simon Riggs wrote:
    - Switching to single-major-version release numbering. The number of
    people who say "PostgreSQL 9.x" is amazing; even *packagers* get this
    wrong and produce "postgresql-9" packages. Witness Amazon Linux's awful
    PostgreSQL packages for example. Going to PostgreSQL 10.0, 11.0, 12.0,
    etc with a typical major/minor scheme might be worth considering.
    In this case you don't even need the 2nd digit...
    You do -- they are used for minor releases, i.e. 10.1 would be a bugfix
    release for 10.0. If we continue using the current numbering scheme,
    10.1 would be the major version after 10.0.
    Btw, -1 for the idea, as it would remove the possibility to tell that a new
    major release incrementing the 1st digit of version number brings more
    enhancement than normal major releases incrementing the 1st digit. This was
    the case for 9.0, helping people in remembering that streaming replication
    has been introduced from 9.x series.
    All major releases bring lots of enhancements. Streaming replication
    might be great for some people, but I'm sure there are things in 8.4 and
    9.1 that are equally great for some other people.

    --
    Álvaro Herrera http://www.2ndQuadrant.com/
    PostgreSQL Development, 24x7 Support, Training & Services
  • Michael Paquier at May 27, 2013 at 10:39 pm

    On Tue, May 28, 2013 at 12:36 AM, Alvaro Herrera wrote:

    Michael Paquier escribió:
    On Mon, May 27, 2013 at 2:01 PM, Craig Ringer wrote:

    On 05/25/2013 05:39 PM, Simon Riggs wrote:
    - Switching to single-major-version release numbering. The number of
    people who say "PostgreSQL 9.x" is amazing; even *packagers* get this
    wrong and produce "postgresql-9" packages. Witness Amazon Linux's awful
    PostgreSQL packages for example. Going to PostgreSQL 10.0, 11.0, 12.0,
    etc with a typical major/minor scheme might be worth considering.
    In this case you don't even need the 2nd digit...
    You do -- they are used for minor releases, i.e. 10.1 would be a bugfix
    release for 10.0. If we continue using the current numbering scheme,
    10.1 would be the major version after 10.0.
    Sorry for the confusion. I meant that the 2nd digit would not be necessary
    when identifying a given major release, so I just didn't get the meaning of
    what Craig said. As you say, you would still need the 2nd digit for minor
    releases.
    --
    Michael
  • David Fetter at May 27, 2013 at 10:52 pm

    On Tue, May 28, 2013 at 07:39:35AM +0900, Michael Paquier wrote:
    On Tue, May 28, 2013 at 12:36 AM, Alvaro Herrera
    wrote:
    Michael Paquier escribió:
    On Mon, May 27, 2013 at 2:01 PM, Craig Ringer <craig@2ndquadrant.com>
    wrote:
    On 05/25/2013 05:39 PM, Simon Riggs wrote:
    - Switching to single-major-version release numbering. The number of
    people who say "PostgreSQL 9.x" is amazing; even *packagers* get this
    wrong and produce "postgresql-9" packages. Witness Amazon Linux's awful
    PostgreSQL packages for example. Going to PostgreSQL 10.0, 11.0, 12.0,
    etc with a typical major/minor scheme might be worth considering.
    In this case you don't even need the 2nd digit...
    You do -- they are used for minor releases, i.e. 10.1 would be a bugfix
    release for 10.0. If we continue using the current numbering scheme,
    10.1 would be the major version after 10.0.
    Sorry for the confusion. I meant that the 2nd digit would not be necessary
    when identifying a given major release, so I just didn't get the meaning of
    what Craig said. As you say, you would still need the 2nd digit for minor
    releases.
    What's been proposed before that wouldn't break previous applications
    is a numbering system like this:

    10.0.0
    10.0.1
    10.0.2
    10.0.3
    ...
    11.0.0
    11.0.1

    i.e. only change the "most-major" version number and always leave the
    "less-major" number as zero.

    Cheers,
    David.
    --
    David Fetter <david@fetter.org> http://fetter.org/
    Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter
    Skype: davidfetter XMPP: david.fetter@gmail.com
    iCal: webcal://www.tripit.com/feed/ical/people/david74/tripit.ics

    Remember to vote!
    Consider donating to Postgres: http://www.postgresql.org/about/donate
  • Michael Paquier at May 27, 2013 at 11:22 pm

    On Tue, May 28, 2013 at 7:52 AM, David Fetter wrote:

    What's been proposed before that wouldn't break previous applications
    is a numbering system like this:

    10.0.0
    10.0.1
    10.0.2
    10.0.3
    ...
    11.0.0
    11.0.1

    i.e. only change the "most-major" version number and always leave the
    "less-major" number as zero.
    Thanks for the clarification. Firefox did exactly the same from 4.0.
    --
    Michael
  • Craig Ringer at May 27, 2013 at 11:54 pm

    On 05/28/2013 07:22 AM, Michael Paquier wrote:
    On Tue, May 28, 2013 at 7:52 AM, David Fetter wrote:

    What's been proposed before that wouldn't break previous applications
    is a numbering system like this:

    10.0.0
    10.0.1
    10.0.2
    10.0.3
    ...
    11.0.0
    11.0.1

    i.e. only change the "most-major" version number and always leave the
    "less-major" number as zero.
    Thanks for the clarification. Firefox did exactly the same from 4.0.
    Yeah... I was more meaning 10.0, 10.1, 10.2 etc for minor releases, but
    I can imagine people coding logic to check "major version" using the
    first two digits, so you're quite right that it'd need to be
    grandfathered into 10.0.1, 10.0.2, etc. Sigh.

    The upside of that is that it'd reinforce the idea that we sometimes
    struggle to get across to people - that minor patch releases are *minor*
    and *safe* to just upgrade to without jumping through change-approval
    hoops, vendor approval for updates, two-year-long QA and all the other
    baggage many IT departments seem to have.

    --
      Craig Ringer http://www.2ndQuadrant.com/
      PostgreSQL Development, 24x7 Support, Training & Services
  • Alvaro Herrera at May 27, 2013 at 11:03 pm

    Michael Paquier escribió:
    On Tue, May 28, 2013 at 12:36 AM, Alvaro Herrera
    wrote:
    You do -- they are used for minor releases, i.e. 10.1 would be a bugfix
    release for 10.0. If we continue using the current numbering scheme,
    10.1 would be the major version after 10.0.
    Sorry for the confusion. I meant that the 2nd digit would not be necessary
    when identifying a given major release, so I just didn't get the meaning of
    what Craig said. As you say, you would still need the 2nd digit for minor
    releases.
    Well, that seems okay to me. We used to see a lot of people talking
    about "Postgres 8.x" when they meant, say, 8.3; and now we have people
    talking about "Postgres 9" when in reality they mean 9.1 or some other
    specific major version. Having the second digit be part of the major
    version number is a difficult idea to convey to external people.

    --
    Álvaro Herrera http://www.2ndQuadrant.com/
    PostgreSQL Development, 24x7 Support, Training & Services
  • Craig Ringer at May 27, 2013 at 11:48 pm

    On 05/27/2013 05:45 PM, Michael Paquier wrote:
    On Mon, May 27, 2013 at 2:01 PM, Craig Ringer wrote:

    On 05/25/2013 05:39 PM, Simon Riggs wrote:
    - Switching to single-major-version release numbering. The number of
    people who say "PostgreSQL 9.x" is amazing; even *packagers* get this
    wrong and produce "postgresql-9" packages. Witness Amazon Linux's awful
    PostgreSQL packages for example. Going to PostgreSQL 10.0, 11.0, 12.0,
    etc with a typical major/minor scheme might be worth considering.
    In this case you don't even need the 2nd digit...
    Btw, -1 for the idea, as it would remove the possibility to tell that a new
    major release incrementing the 1st digit of version number brings more
    enhancement than normal major releases incrementing the 1st digit. This was
    the case for 9.0, helping people in remembering that streaming replication
    has been introduced from 9.x series.
    I don't find bumping the major to be particularly helpful. Every
    release brings major features - and some introduce major incompatibilities.

    8.4 introduced CTEs.
    8.3 broke tons of client code with the removal of implicit casts to text.

    It really depends on what features you consider more major/significant.
    Personally I don't think it makes sense to try to say "this release is
    bigger" in Pg - at least not in terms of enhancement. I can see value in
    using this-release-is-bigger for "this brings more breakage" - but would
    strongly prefer a smooth and continuous release numbering that doesn't
    confuse the heck out of users.

    I'm extremely tired of being told "I'm running PostgreSQL 8.x" or "I'm
    running PostgreSQL 9.x" and having to point out the version policy, the
    fact that there are four years and huge fixes/enhancements between 8.0
    and 8.4, etc.

    The version policy makes _no distinction_ between which digit changes in
    a major release:

    "PostgreSQL major releases include new features and occur roughly once
    every year. A major release is numbered by increasing either the first
    or second part of the version number, e.g. 8.2 to 8.3.

    "Major releases usually change the internal format of system tables and
    data files. These changes are often complex, so we do not maintain
    backward compatibility of all stored data. A dump/reload of the database
    or use of the pg_upgrade module is required for major upgrades."

    and I strongly believe that we should drop the notion entirely.

    ...

    --
      Craig Ringer http://www.2ndQuadrant.com/
      PostgreSQL Development, 24x7 Support, Training & Services
  • Gavin Flower at May 28, 2013 at 1:40 am

    On 28/05/13 11:48, Craig Ringer wrote:
    On 05/27/2013 05:45 PM, Michael Paquier wrote:
    On Mon, May 27, 2013 at 2:01 PM, Craig Ringerwrote:
    On 05/25/2013 05:39 PM, Simon Riggs wrote:
    - Switching to single-major-version release numbering. The number of
    people who say "PostgreSQL 9.x" is amazing; even *packagers* get this
    wrong and produce "postgresql-9" packages. Witness Amazon Linux's awful
    PostgreSQL packages for example. Going to PostgreSQL 10.0, 11.0, 12.0,
    etc with a typical major/minor scheme might be worth considering.
    In this case you don't even need the 2nd digit...
    Btw, -1 for the idea, as it would remove the possibility to tell that a new
    major release incrementing the 1st digit of version number brings more
    enhancement than normal major releases incrementing the 1st digit. This was
    the case for 9.0, helping people in remembering that streaming replication
    has been introduced from 9.x series.
    I don't find bumping the major to be particularly helpful. Every
    release brings major features - and some introduce major
    incompatibilities.

    8.4 introduced CTEs.
    8.3 broke tons of client code with the removal of implicit casts to text.

    It really depends on what features you consider more
    major/significant. Personally I don't think it makes sense to try to
    say "this release is bigger" in Pg - at least not in terms of
    enhancement. I can see value in using this-release-is-bigger for "this
    brings more breakage" - but would strongly prefer a smooth and
    continuous release numbering that doesn't confuse the heck out of users.

    I'm extremely tired of being told "I'm running PostgreSQL 8.x" or "I'm
    running PostgreSQL 9.x" and having to point out the version policy,
    the fact that there are four years and huge fixes/enhancements between
    8.0 and 8.4, etc.

    The version policy makes _no distinction_ between which digit changes
    in a major release:

    "PostgreSQL major releases include new features and occur roughly once
    every year. A major release is numbered by increasing either the first
    or second part of the version number, e.g. 8.2 to 8.3.

    "Major releases usually change the internal format of system tables
    and data files. These changes are often complex, so we do not maintain
    backward compatibility of all stored data. A dump/reload of the
    database or use of the pg_upgrade module is required for major upgrades."

    and I strongly believe that we should drop the notion entirely.

    ...

    --
    Craig Ringerhttp://www.2ndQuadrant.com/
    PostgreSQL Development, 24x7 Support, Training & Services
    Yes, I hate the Firefox style number inflation.


    Cheers,
    Gavin

Related Discussions

People

Translate

site design / logo © 2021 Grokbase