Hi all!

It seems that the git move has broken the generation of the automatic
snapshot tarballs - has anybody yet looked into what it would take to
move those to fetching from git?


Stefan

Search Discussions

  • Heikki Linnakangas at Sep 22, 2010 at 8:47 am

    On 22/09/10 11:33, Stefan Kaltenbrunner wrote:
    Hi all!

    It seems that the git move has broken the generation of the automatic
    snapshot tarballs - has anybody yet looked into what it would take to
    move those to fetching from git?
    BTW, there is a nice command called in git to create tarballs:

    git archive master | gzip > postgresql.tar.gz

    I doubt we can use it because we add some generated files to our
    tarballs that are not in the repository, but I thought I'd mention it.

    --
    Heikki Linnakangas
    EnterpriseDB http://www.enterprisedb.com
  • Magnus Hagander at Sep 22, 2010 at 8:49 am

    On Wed, Sep 22, 2010 at 10:47, Heikki Linnakangas wrote:
    On 22/09/10 11:33, Stefan Kaltenbrunner wrote:

    Hi all!

    It seems that the git move has broken the generation of the automatic
    snapshot tarballs - has anybody yet looked into what it would take to
    move those to fetching from git?
    BTW, there is a nice command called in git to create tarballs:

    git archive master | gzip > postgresql.tar.gz

    I doubt we can use it because we add some generated files to our tarballs
    that are not in the repository, but I thought I'd mention it.
    It is useful, yes, but not for this.

    I'll take a peek at the tarball generation today, unless beaten to it
    (meaning if someone wants to star tlooking into it, send me an email
    or ping me on irc to make sure we don't duplicate the work)
  • Peter Eisentraut at Sep 22, 2010 at 9:29 am

    On ons, 2010-09-22 at 10:33 +0200, Stefan Kaltenbrunner wrote:
    It seems that the git move has broken the generation of the automatic
    snapshot tarballs - has anybody yet looked into what it would take to
    move those to fetching from git?
    Depends on what's "broken" about it, but I notice that the developer
    docs and the NLS builds are also not updating. Perhaps something wrong
    with the anoncvs service.
  • Magnus Hagander at Sep 22, 2010 at 10:12 am

    On Wed, Sep 22, 2010 at 11:29, Peter Eisentraut wrote:
    On ons, 2010-09-22 at 10:33 +0200, Stefan Kaltenbrunner wrote:
    It seems that the git move has broken the generation of the automatic
    snapshot tarballs - has anybody yet looked into what it would take to
    move those to fetching from git?
    Depends on what's "broken" about it, but I notice that the developer
    docs and the NLS builds are also not updating.  Perhaps something wrong
    with the anoncvs service.
    anoncvs still serves the old cvs repository.
  • Dave Page at Sep 22, 2010 at 10:19 am

    On Wed, Sep 22, 2010 at 11:11 AM, Magnus Hagander wrote:
    On Wed, Sep 22, 2010 at 11:29, Peter Eisentraut wrote:
    On ons, 2010-09-22 at 10:33 +0200, Stefan Kaltenbrunner wrote:
    It seems that the git move has broken the generation of the automatic
    snapshot tarballs - has anybody yet looked into what it would take to
    move those to fetching from git?
    Depends on what's "broken" about it, but I notice that the developer
    docs and the NLS builds are also not updating.  Perhaps something wrong
    with the anoncvs service.
    anoncvs still serves the old cvs repository.
    Whens that due to be fixed? I imagine most of the buildfarm is testing
    that still...



    --
    Dave Page
    Blog: http://pgsnake.blogspot.com
    Twitter: @pgsnake

    EnterpriseDB UK: http://www.enterprisedb.com
    The Enterprise Postgres Company
  • Magnus Hagander at Sep 22, 2010 at 10:22 am

    On Wed, Sep 22, 2010 at 12:19, Dave Page wrote:
    On Wed, Sep 22, 2010 at 11:11 AM, Magnus Hagander wrote:
    On Wed, Sep 22, 2010 at 11:29, Peter Eisentraut wrote:
    On ons, 2010-09-22 at 10:33 +0200, Stefan Kaltenbrunner wrote:
    It seems that the git move has broken the generation of the automatic
    snapshot tarballs - has anybody yet looked into what it would take to
    move those to fetching from git?
    Depends on what's "broken" about it, but I notice that the developer
    docs and the NLS builds are also not updating.  Perhaps something wrong
    with the anoncvs service.
    anoncvs still serves the old cvs repository.
    Whens that due to be fixed? I imagine most of the buildfarm is testing
    that still...
    Don't have an actual time for that. Figured we'd leave the old one
    running until things have settled down.

    Mind you, I think bf members will need tweaking anyway, since they
    will assume the old style cvs layout. AFAICT, git-cvsserver will
    export each branch as a module, rather than one module as pgsql. We
    can obviously get that working for the HEAD branch (by just mapping it
    to a branch called pgsql), but I don't know how doable that is for
    backbranches.
  • Peter Eisentraut at Sep 22, 2010 at 11:20 am

    On ons, 2010-09-22 at 12:22 +0200, Magnus Hagander wrote:
    Mind you, I think bf members will need tweaking anyway, since they
    will assume the old style cvs layout. AFAICT, git-cvsserver will
    export each branch as a module, rather than one module as pgsql. We
    can obviously get that working for the HEAD branch (by just mapping it
    to a branch called pgsql), but I don't know how doable that is for
    backbranches.
    We need to have a fix for this before we can do any more backbranch
    releases.
  • Magnus Hagander at Sep 22, 2010 at 11:25 am

    On Wed, Sep 22, 2010 at 13:20, Peter Eisentraut wrote:
    On ons, 2010-09-22 at 12:22 +0200, Magnus Hagander wrote:
    Mind you, I think bf members will need tweaking anyway, since they
    will assume the old style cvs layout. AFAICT, git-cvsserver will
    export each branch as a module, rather than one module as pgsql. We
    can obviously get that working for the HEAD branch (by just mapping it
    to a branch called pgsql), but I don't know how doable that is for
    backbranches.
    We need to have a fix for this before we can do any more backbranch
    releases.
    Fix for what, exactly? (assuming that people update their bf animals of course)
  • Stefan Kaltenbrunner at Sep 22, 2010 at 11:29 am

    Magnus Hagander wrote:
    On Wed, Sep 22, 2010 at 13:20, Peter Eisentraut wrote:
    On ons, 2010-09-22 at 12:22 +0200, Magnus Hagander wrote:
    Mind you, I think bf members will need tweaking anyway, since they
    will assume the old style cvs layout. AFAICT, git-cvsserver will
    export each branch as a module, rather than one module as pgsql. We
    can obviously get that working for the HEAD branch (by just mapping it
    to a branch called pgsql), but I don't know how doable that is for
    backbranches.
    We need to have a fix for this before we can do any more backbranch
    releases.
    Fix for what, exactly? (assuming that people update their bf animals of course)
    how exactly should people fix their BF members - is there any howto or
    such on how to deal with the new git2cvs gateway?
    Switching to git entirely is rather hard for some of my members for
    example - at least not without investing hours of work...


    Stefan
  • Magnus Hagander at Sep 22, 2010 at 11:44 am

    On Wed, Sep 22, 2010 at 13:28, Stefan Kaltenbrunner wrote:
    Magnus Hagander wrote:
    On Wed, Sep 22, 2010 at 13:20, Peter Eisentraut wrote:
    On ons, 2010-09-22 at 12:22 +0200, Magnus Hagander wrote:

    Mind you, I think bf members will need tweaking anyway, since they
    will assume the old style cvs layout. AFAICT, git-cvsserver will
    export each branch as a module, rather than one module as pgsql. We
    can obviously get that working for the HEAD branch (by just mapping it
    to a branch called pgsql), but I don't know how doable that is for
    backbranches.
    We need to have a fix for this before we can do any more backbranch
    releases.
    Fix for what, exactly? (assuming that people update their bf animals of
    course)
    how exactly should people fix their BF members - is there any howto or such
    on how to deal with the new git2cvs gateway?
    Andrew has posted info on how to convert them to git, I believe.

    I don't think there are any for the git2cvs gateway, since it's not up yet.
  • Peter Eisentraut at Sep 22, 2010 at 11:38 am

    On ons, 2010-09-22 at 13:25 +0200, Magnus Hagander wrote:
    On Wed, Sep 22, 2010 at 13:20, Peter Eisentraut wrote:
    We need to have a fix for this before we can do any more backbranch
    releases.
    Fix for what, exactly? (assuming that people update their bf animals of course)
    For the NLS workflow, we need to have some kind of CVS server that
    serves up all supported branches. Changing the URL or the module name
    or whatever is fine.
  • Magnus Hagander at Sep 22, 2010 at 11:45 am

    On Wed, Sep 22, 2010 at 13:38, Peter Eisentraut wrote:
    On ons, 2010-09-22 at 13:25 +0200, Magnus Hagander wrote:
    On Wed, Sep 22, 2010 at 13:20, Peter Eisentraut wrote:
    We need to have a fix for this before we can do any more backbranch
    releases.
    Fix for what, exactly? (assuming that people update their bf animals of course)
    For the NLS workflow, we need to have some kind of CVS server that
    serves up all supported branches.  Changing the URL or the module name
    or whatever is fine.
    I thought you had/were going to convert those to work natively with
    git? I guess I misunderstood you.

    Anyway, it's on my list to get done pretty quickly.
  • Peter Eisentraut at Sep 22, 2010 at 11:54 am

    On ons, 2010-09-22 at 13:45 +0200, Magnus Hagander wrote:
    On Wed, Sep 22, 2010 at 13:38, Peter Eisentraut wrote:
    For the NLS workflow, we need to have some kind of CVS server that
    serves up all supported branches. Changing the URL or the module name
    or whatever is fine.
    I thought you had/were going to convert those to work natively with
    git? I guess I misunderstood you.
    I think CVS should be eliminated from that process during the 9.1
    release cycle, but we are looking for backbranch releases within the
    next few weeks, and there is no time to get that done by then.
  • Magnus Hagander at Sep 22, 2010 at 1:09 pm

    On Wed, Sep 22, 2010 at 13:45, Magnus Hagander wrote:
    On Wed, Sep 22, 2010 at 13:38, Peter Eisentraut wrote:
    On ons, 2010-09-22 at 13:25 +0200, Magnus Hagander wrote:
    On Wed, Sep 22, 2010 at 13:20, Peter Eisentraut wrote:
    We need to have a fix for this before we can do any more backbranch
    releases.
    Fix for what, exactly? (assuming that people update their bf animals of course)
    For the NLS workflow, we need to have some kind of CVS server that
    serves up all supported branches.  Changing the URL or the module name
    or whatever is fine.
    I thought you had/were going to convert those to work natively with
    git? I guess I misunderstood you.

    Anyway, it's on my list to get done pretty quickly.
    Started again on this.

    Whenever I try to do something useful, it crashes with sqlite errors
    that appear to be FreeBSD specific.

    I'm going to try to upgrade all the ports on the box and give it
    another try. Unfortunately, that tends to take a couple of hours - but
    it'll get done eventually :-)
  • Stefan Kaltenbrunner at Sep 22, 2010 at 11:27 am

    Peter Eisentraut wrote:
    On ons, 2010-09-22 at 12:22 +0200, Magnus Hagander wrote:
    Mind you, I think bf members will need tweaking anyway, since they
    will assume the old style cvs layout. AFAICT, git-cvsserver will
    export each branch as a module, rather than one module as pgsql. We
    can obviously get that working for the HEAD branch (by just mapping it
    to a branch called pgsql), but I don't know how doable that is for
    backbranches.
    We need to have a fix for this before we can do any more backbranch
    releases.
    yeah having to touch ever BF member despite the fact that we have a
    git->cvs gateway seems heavily annoying and not really expected. However
    if this must be done oen should send a proper headsup and howto to the
    buildfarm member list (people need to add the 9.0 branch as well so we
    better tell them this now instead of having to do work twice).



    Stefan
  • Andrew Dunstan at Sep 22, 2010 at 12:00 pm

    On 09/22/2010 06:22 AM, Magnus Hagander wrote:
    Mind you, I think bf members will need tweaking anyway, since they
    will assume the old style cvs layout. AFAICT, git-cvsserver will
    export each branch as a module, rather than one module as pgsql. We
    can obviously get that working for the HEAD branch (by just mapping it
    to a branch called pgsql), but I don't know how doable that is for
    backbranches.
    Buildfarm owners should be moving aggressively towards using git. I'm
    not aware of any platform it can't be used on.

    I was planning on doing an audit around the end of the week and
    following up with people who haven't migrated.

    cheers

    andrew
  • Stefan Kaltenbrunner at Sep 22, 2010 at 10:19 am

    Peter Eisentraut wrote:
    On ons, 2010-09-22 at 10:33 +0200, Stefan Kaltenbrunner wrote:
    It seems that the git move has broken the generation of the automatic
    snapshot tarballs - has anybody yet looked into what it would take to
    move those to fetching from git?
    Depends on what's "broken" about it, but I notice that the developer
    docs and the NLS builds are also not updating. Perhaps something wrong
    with the anoncvs service.
    heh - well nagios complained that the current tarballs are outdated, I
    have not actually investigated further :)
    But iirc snapshot generation is running against the main cvs repo and
    not anoncvs.

    Stefan
  • Peter Eisentraut at Sep 23, 2010 at 8:08 am

    On ons, 2010-09-22 at 12:29 +0300, Peter Eisentraut wrote:
    On ons, 2010-09-22 at 10:33 +0200, Stefan Kaltenbrunner wrote:
    It seems that the git move has broken the generation of the automatic
    snapshot tarballs - has anybody yet looked into what it would take to
    move those to fetching from git?
    Depends on what's "broken" about it, but I notice that the developer
    docs and the NLS builds are also not updating. Perhaps something wrong
    with the anoncvs service.
    Developer docs are now building again.
  • Peter Eisentraut at Sep 24, 2010 at 5:16 am

    On tor, 2010-09-23 at 11:07 +0300, Peter Eisentraut wrote:
    On ons, 2010-09-22 at 12:29 +0300, Peter Eisentraut wrote:
    On ons, 2010-09-22 at 10:33 +0200, Stefan Kaltenbrunner wrote:
    It seems that the git move has broken the generation of the automatic
    snapshot tarballs - has anybody yet looked into what it would take to
    move those to fetching from git?
    Depends on what's "broken" about it, but I notice that the developer
    docs and the NLS builds are also not updating. Perhaps something wrong
    with the anoncvs service.
    Developer docs are now building again.
    And NLS is also fixed.
  • Magnus Hagander at Sep 24, 2010 at 8:27 am

    On Fri, Sep 24, 2010 at 07:15, Peter Eisentraut wrote:
    On tor, 2010-09-23 at 11:07 +0300, Peter Eisentraut wrote:
    On ons, 2010-09-22 at 12:29 +0300, Peter Eisentraut wrote:
    On ons, 2010-09-22 at 10:33 +0200, Stefan Kaltenbrunner wrote:
    It seems that the git move has broken the generation of the automatic
    snapshot tarballs - has anybody yet looked into what it would take to
    move those to fetching from git?
    Depends on what's "broken" about it, but I notice that the developer
    docs and the NLS builds are also not updating.  Perhaps something wrong
    with the anoncvs service.
    Developer docs are now building again.
    And NLS is also fixed.
    Great. Thanks - that takes one more thing off the cvs requirement ;)
  • Tom Lane at Sep 24, 2010 at 7:47 pm

    Magnus Hagander writes:
    On Fri, Sep 24, 2010 at 07:15, Peter Eisentraut wrote:
    And NLS is also fixed.
    Great. Thanks - that takes one more thing off the cvs requirement ;)
    Yeah. Maybe we don't need a cvsserver after all. Would it make more
    sense to help Stefan get git running on his BSD boxes? If Bruce and
    I could get it to work on our pet dinosaurs, I think it likely can
    be gotten to work on spoonbill too.

    regards, tom lane
  • Andrew Dunstan at Sep 24, 2010 at 7:54 pm

    On 09/24/2010 03:47 PM, Tom Lane wrote:

    Yeah. Maybe we don't need a cvsserver after all. Would it make more
    sense to help Stefan get git running on his BSD boxes? If Bruce and
    I could get it to work on our pet dinosaurs, I think it likely can
    be gotten to work on spoonbill too.
    Yeah, I find it hard to believe that a machine can build postgres, with
    openssl, but can't build git. The problem is that Stefan doesn't have
    time to work on it, and I gather he's unable to give anyone else access.

    cheers

    andrew
  • Stefan Kaltenbrunner at Sep 24, 2010 at 9:34 pm

    On 09/24/2010 09:54 PM, Andrew Dunstan wrote:
    On 09/24/2010 03:47 PM, Tom Lane wrote:

    Yeah. Maybe we don't need a cvsserver after all. Would it make more
    sense to help Stefan get git running on his BSD boxes? If Bruce and
    I could get it to work on our pet dinosaurs, I think it likely can
    be gotten to work on spoonbill too.
    Yeah, I find it hard to believe that a machine can build postgres, with
    openssl, but can't build git. The problem is that Stefan doesn't have
    time to work on it, and I gather he's unable to give anyone else access.
    yeah don't worry too much, I will find a way to fix it will just take a
    few more days until I get a bit of spare time...


    Stefan
  • Magnus Hagander at Sep 22, 2010 at 10:25 am

    On Wed, Sep 22, 2010 at 10:33, Stefan Kaltenbrunner wrote:
    Hi all!

    It seems that the git move has broken the generation of the automatic
    snapshot tarballs - has anybody yet looked into what it would take to move
    those to fetching from git?
    I think this should be working now. I made this change to mk-snapshot,
    and a similar one to mk-stable_snapshot. I have run the snapshot
    generation for master, and the one for 8.4 should finish in a
    minute or so. I have disabled the cronjobs until someone else has
    verified the tarballs.

    When I do, I'll also enable generation of snapshots of REL9_0_STABLE.
    We currently do back to 8.2 - should we keep that, or drop 8.2?

    //Magnus


    *** old/mk-snapshot Wed Sep 22 06:02:02 2010
    --- mk-snapshot Wed Sep 22 07:09:13 2010
    ***************
    *** 8,14 ****
    export MAKE=gmake
    export MD5SUM=md5

    ! export CVSROOT=:ext:scrappy@cvs.postgresql.org:/cvsroot

    export MAKEFLAGS=VERSION=snapshot
    export TMPDIR=/usr/local/pgsql/snapshot
    --- 8,14 ----
    export MAKE=gmake
    export MD5SUM=md5

    ! export GIT_DIR=/usr/local/pgsql/git/postgresql/.git

    export MAKEFLAGS=VERSION=snapshot
    export TMPDIR=/usr/local/pgsql/snapshot
    ***************
    *** 19,25 ****
    then
    rm -fr pgsql
    fi
    ! /usr/bin/cvs -q export -rHEAD pgsql
    cd pgsql
    ./configure
    export OUTPUTFILE=$($MAKE -s $MAKEFLAGS distdir-location)
    --- 19,28 ----
    then
    rm -fr pgsql
    fi
    ! mkdir pgsql
    ! git fetch
    ! git archive origin/master | tar xf - -C pgsql
    !
    cd pgsql
    ./configure
    export OUTPUTFILE=$($MAKE -s $MAKEFLAGS distdir-location)
  • Stefan Kaltenbrunner at Sep 22, 2010 at 10:28 am

    Magnus Hagander wrote:
    On Wed, Sep 22, 2010 at 10:33, Stefan Kaltenbrunner
    wrote:
    Hi all!

    It seems that the git move has broken the generation of the automatic
    snapshot tarballs - has anybody yet looked into what it would take to move
    those to fetching from git?
    I think this should be working now. I made this change to mk-snapshot,
    and a similar one to mk-stable_snapshot. I have run the snapshot
    generation for master, and the one for 8.4 should finish in a
    minute or so. I have disabled the cronjobs until someone else has
    verified the tarballs.

    When I do, I'll also enable generation of snapshots of REL9_0_STABLE.
    We currently do back to 8.2 - should we keep that, or drop 8.2?
    hmm I would say we should do all supported branches because those are
    the ones that people might be interested in when looking for a fix on a
    branch...


    Stefan
  • Tom Lane at Sep 22, 2010 at 2:50 pm

    Stefan Kaltenbrunner writes:
    Magnus Hagander wrote:
    I think this should be working now. I made this change to mk-snapshot,
    and a similar one to mk-stable_snapshot. I have run the snapshot
    generation for master, and the one for 8.4 should finish in a
    minute or so. I have disabled the cronjobs until someone else has
    verified the tarballs.
    Do you still need someone to do that, and what do you want done exactly?
    When I do, I'll also enable generation of snapshots of REL9_0_STABLE.
    We currently do back to 8.2 - should we keep that, or drop 8.2?
    hmm I would say we should do all supported branches because those are
    the ones that people might be interested in when looking for a fix on a
    branch...
    FWIW, I think back to 8.2 is sufficient. The branches earlier than that
    are going to be unsupported in a matter of weeks anyway. And if no one
    has complained about the lack of nightly tarballs for them to date,
    it's unlikely that they'll start complaining now.

    regards, tom lane
  • Magnus Hagander at Sep 22, 2010 at 3:05 pm

    On Wed, Sep 22, 2010 at 16:50, Tom Lane wrote:
    Stefan Kaltenbrunner <stefan@kaltenbrunner.cc> writes:
    Magnus Hagander wrote:
    I think this should be working now. I made this change to mk-snapshot,
    and a similar one to mk-stable_snapshot. I have run the snapshot
    generation for master, and the one for 8.4 should finish in a
    minute or so. I have disabled the cronjobs until someone else has
    verified the tarballs.
    Do you still need someone to do that, and what do you want done exactly?
    Just a second set of eyes that the output looks reasonable for being a
    snapshot generated now.

    When I do, I'll also enable generation of snapshots of REL9_0_STABLE.
    We currently do back to 8.2 - should we keep that, or drop 8.2?
    hmm I would say we should do all supported branches because those are
    the ones that people might be interested in when looking for a fix on a
    branch...
    FWIW, I think back to 8.2 is sufficient.  The branches earlier than that
    are going to be unsupported in a matter of weeks anyway.  And if no one
    has complained about the lack of nightly tarballs for them to date,
    it's unlikely that they'll start complaining now.
    Ok. 8.2 it is then - and I'll add 9.0
  • Tom Lane at Sep 22, 2010 at 3:08 pm

    Magnus Hagander writes:
    On Wed, Sep 22, 2010 at 16:50, Tom Lane wrote:
    Do you still need someone to do that, and what do you want done exactly?
    Just a second set of eyes that the output looks reasonable for being a
    snapshot generated now.
    Sure, I can look. Where are these tarballs anyway?

    regards, tom lane
  • Magnus Hagander at Sep 22, 2010 at 4:50 pm

    On Wed, Sep 22, 2010 at 17:08, Tom Lane wrote:
    Magnus Hagander <magnus@hagander.net> writes:
    On Wed, Sep 22, 2010 at 16:50, Tom Lane wrote:
    Do you still need someone to do that, and what do you want done exactly?
    Just a second set of eyes that the output looks reasonable for being a
    snapshot generated now.
    Sure, I can look.  Where are these tarballs anyway?
    They're up on the ftp site - ftp://ftp.postgresql.org/pub/snapshot/.
    dev and stable/8.4 are updated.
  • Tom Lane at Sep 22, 2010 at 5:23 pm

    Magnus Hagander writes:
    On Wed, Sep 22, 2010 at 17:08, Tom Lane wrote:
    Sure, I can look.  Where are these tarballs anyway?
    They're up on the ftp site - ftp://ftp.postgresql.org/pub/snapshot/.
    dev and stable/8.4 are updated.
    Both of those look pretty sane from here.

    regards, tom lane
  • Magnus Hagander at Sep 22, 2010 at 5:25 pm

    On Wed, Sep 22, 2010 at 19:23, Tom Lane wrote:
    Magnus Hagander <magnus@hagander.net> writes:
    On Wed, Sep 22, 2010 at 17:08, Tom Lane wrote:
    Sure, I can look.  Where are these tarballs anyway?
    They're up on the ftp site - ftp://ftp.postgresql.org/pub/snapshot/.
    dev and stable/8.4 are updated.
    Both of those look pretty sane from here.
    Ok, I've re-enabled the cron job.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouppgsql-hackers @
categoriespostgresql
postedSep 22, '10 at 8:34a
activeSep 24, '10 at 9:34p
posts32
users7
websitepostgresql.org...
irc#postgresql

People

Translate

site design / logo © 2022 Grokbase