FAQ
hi,
as it was told that centos can be reproduced and all rpm build in mock i
just made a quick test and try to rebuild all centos-5.3 srpms. the
result not too bad but far from perfect. the setup was a fresh i386
centos-5.3 minimal install + yum update + install mock-0.6.13-1.el5_2.3
(the result was almost the same even worse with 2 srpms with
mock-0.9.14-1.el5).
from 1186 srpms
- 1145 build ~96.5%
- 41 failed ~ 3.5%
for me it seems to me the failed rpms have the same version as in 5.2
and they was simple copied to 5.3 and was not rebuild for/with 5.3.
the failures have different reasons, just to mention a few:

- aspell-xx use different filename encoding and with the default
en_US.UTF-8 setting it can't find it's files (so those two patches are
strange for me in src.rpm) may be upstream bug.

- axis buildreq install java-1.6.0 (which is in 5.3 but was not in 5.2)
in stead of using gcc's java-1.4.2-gcj-compat-devel. it can be rpm or
yum bug, but can't be rebuild in mock but can be build by hand.

- mysql test suit is failed with:
Errors are (from
/builddir/build/BUILD/mysql-5.0.45/mysql-test/var/log/mysqltest-time) :
mysqltest: Could not open connection 'default': 2026 SSL connection error

the packages which failed to build:

aspell-es-0.50-13.2.2.src.rpm
aspell-no-0.50.1-9.2.2.src.rpm
aspell-pt-0.50-10.2.2.src.rpm
axis-1.2.1-2jpp.6.src.rpm
castor-0.9.5-1jpp.7.src.rpm
cmirror-kmod-0.1.21-10.el5.src.rpm
cyrus-imapd-2.3.7-2.el5.src.rpm
dbus-python-0.70-7.el5.src.rpm
geronimo-specs-1.0-0.M2.2jpp.12.el5.centos.src.rpm
gfs-kmod-0.1.31-3.el5.src.rpm
gjdoc-0.7.7-12.el5.src.rpm
gnbd-kmod-0.1.5-2.el5.centos.src.rpm
gnome-media-2.16.1-3.el5.src.rpm
gnome-pilot-2.0.13-16.src.rpm
gpm-1.20.1-74.1.src.rpm
ifd-egate-0.05-15.src.rpm
junit-3.8.2-3jpp.1.src.rpm
kdebase-3.5.4-19.el5.centos.1.src.rpm
kdebindings-3.5.4-6.el5.src.rpm
libgssapi-0.10-2.src.rpm
libwvstreams-4.2.2-2.1.src.rpm
lsof-4.78-3.src.rpm
mikmod-3.1.6-38.1.src.rpm
mvapich-1.1.0-0.2931.3.el5.src.rpm
mysql-5.0.45-7.el5.src.rpm
nautilus-open-terminal-0.6-6.el5.src.rpm
net-snmp-5.3.2.2-5.el5.src.rpm
nfs-utils-lib-1.0.8-7.2.z2.src.rpm
notification-daemon-0.3.5-9.el5.src.rpm
openhpi-2.10.2-1.el5.src.rpm
perl-XML-Simple-2.14-4.fc6.src.rpm
php-pear-1.4.9-4.el5.1.src.rpm
postgresql-jdbc-8.1.407-1jpp.4.src.rpm
python-docs-2.4.3-1.1.src.rpm
sblim-1-31.el5_2.1.src.rpm
stardict-2.4.5-5.src.rpm
struts-1.2.9-4jpp.5.src.rpm
tetex-3.0-33.2.el5_1.2.src.rpm
totem-2.16.7-4.el5.src.rpm
velocity-1.4-6jpp.1.src.rpm
xdoclet-1.2.3-7jpp.2.src.rpm


--
Levente "Si vis pacem para bellum!"

Search Discussions

  • Jeff Johnson at Apr 20, 2009 at 4:27 pm

    On Apr 20, 2009, at 11:57 AM, Farkas Levente wrote:
    - mysql test suit is failed with:
    Errors are (from
    /builddir/build/BUILD/mysql-5.0.45/mysql-test/var/log/mysqltest-
    time) :
    mysqltest: Could not open connection 'default': 2026 SSL connection
    error
    There's a hack to disable the %check section (if needed):

    Put this in some macro file (e.g. /etc/rpm/macros or ~/.rpmmacros)

    %check %%check \\
    exit 0

    One can also do on the CLI using --define, but multiline escaped
    macro definitions are likely easier done in some macros file.

    One can also disable %check (or any other section of a build)
    by overloading the scriptlet templates to put "exit 0" into the
    pre template.

    E.g. here's the templates for %check (all stubbed with defaults):

    %__spec_check_shell %{___build_shell}
    %__spec_check_args %{___build_args}
    %__spec_check_cmd %{___build_cmd}
    %__spec_check_pre %{___build_pre}
    %__spec_check_body %{___build_body}
    %__spec_check_post %{___build_post}
    %__spec_check_template #!%{__spec_check_shell}\
    %{__spec_check_pre}\
    %{nil}

    So one can bury an "exit 0" into the %check pre section with this
    config:

    %__spec_check_pre exit 0

    How a build system is configured (or rpmbuild arguments used when
    invoking)
    is what makes the "reproducible build" lie pleasant.

    If only 41 (~3.5%) package build failures, that's not too shabby.

    hth

    73 de Jeff
  • Farkas Levente at Apr 20, 2009 at 9:26 pm

    Jeff Johnson wrote:
    On Apr 20, 2009, at 11:57 AM, Farkas Levente wrote:
    - mysql test suit is failed with:
    Errors are (from
    /builddir/build/BUILD/mysql-5.0.45/mysql-test/var/log/mysqltest-
    time) :
    mysqltest: Could not open connection 'default': 2026 SSL connection
    error
    There's a hack to disable the %check section (if needed):

    Put this in some macro file (e.g. /etc/rpm/macros or ~/.rpmmacros)

    %check %%check \\
    exit 0

    One can also do on the CLI using --define, but multiline escaped
    macro definitions are likely easier done in some macros file.

    One can also disable %check (or any other section of a build)
    by overloading the scriptlet templates to put "exit 0" into the
    pre template.

    E.g. here's the templates for %check (all stubbed with defaults):

    %__spec_check_shell %{___build_shell}
    %__spec_check_args %{___build_args}
    %__spec_check_cmd %{___build_cmd}
    %__spec_check_pre %{___build_pre}
    %__spec_check_body %{___build_body}
    %__spec_check_post %{___build_post}
    %__spec_check_template #!%{__spec_check_shell}\
    %{__spec_check_pre}\
    %{nil}

    So one can bury an "exit 0" into the %check pre section with this
    config:

    %__spec_check_pre exit 0

    How a build system is configured (or rpmbuild arguments used when
    invoking)
    is what makes the "reproducible build" lie pleasant.
    unfortunately
    make check
    is in the %build section of mysql spec file so this can't help:-(
    can we know the centos's build environments? ie. mock config, rpm macros
    etc.

    --
    Levente "Si vis pacem para bellum!"
  • Farkas Levente at Apr 23, 2009 at 1:06 pm

    Jeff Johnson wrote:
    On Apr 20, 2009, at 11:57 AM, Farkas Levente wrote:
    - mysql test suit is failed with:
    Errors are (from
    /builddir/build/BUILD/mysql-5.0.45/mysql-test/var/log/mysqltest-
    time) :
    mysqltest: Could not open connection 'default': 2026 SSL connection
    error
    There's a hack to disable the %check section (if needed):

    Put this in some macro file (e.g. /etc/rpm/macros or ~/.rpmmacros)

    %check %%check \\
    exit 0
    strange it's not working for me:-( rpmbuild still run the %check section
    and still exit with error.

    ps. anyway it's turn out that the upstream src.rpm contains expired ssl
    cert which is the reason why it's not longer rebuildable:-)

    --
    Levente "Si vis pacem para bellum!"
  • Jeff Johnson at Apr 23, 2009 at 1:51 pm

    On Apr 23, 2009, at 9:06 AM, Farkas Levente wrote:
    strange it's not working for me:-( rpmbuild still run the %check
    section
    and still exit with error.
    All -debuginfo packages are produced by overloading
    a macro to insert text into a spec file in the same fashion.

    But yes, its a hack, just like -debuginfo packages are a hack.
    ps. anyway it's turn out that the upstream src.rpm contains expired
    ssl
    cert which is the reason why it's not longer rebuildable:-)
    Glad you figgered.

    73 de Jeff
  • R P Herrold at Apr 20, 2009 at 4:42 pm

    On Mon, 20 Apr 2009, Farkas Levente wrote:

    as it was told that centos can be reproduced and all rpm build in mock i
    just made a quick test and try to rebuild all centos-5.3 srpms. the
    - mysql test suite is failed with:
    Errors are (from
    /builddir/build/BUILD/mysql-5.0.45/mysql-test/var/log/mysqltest-time) :
    mysqltest: Could not open connection 'default': 2026 SSL connection error
    That would seem to be a deficit in the build environment, not
    starting the candidate mysql-server to connect against; I
    would check your buildenv, and verify that a target to connect
    to was present; that is, see what the process table holds,
    and to what the make test is trying to connect.
    the packages which failed to build:
    (snip list)

    Here, checking the build logs in a postmortem, and tweaking in
    what may be missing is a first step.

    -- Russ herrold
  • David Hrbáč at Apr 24, 2009 at 2:31 pm

    Farkas Levente napsal(a):
    hi,
    as it was told that centos can be reproduced and all rpm build in mock i
    just made a quick test and try to rebuild all centos-5.3 srpms. the
    result not too bad but far from perfect. the setup was a fresh i386
    centos-5.3 minimal install + yum update + install mock-0.6.13-1.el5_2.3
    (the result was almost the same even worse with 2 srpms with
    mock-0.9.14-1.el5).
    from 1186 srpms
    Hi Farkas,
    what is you method to obtain rmp build list with correct dependencies?
    Could you share how you automate it?
    Thanks,
    David
  • Jeff Johnson at Apr 24, 2009 at 2:40 pm

    On Apr 24, 2009, at 10:31 AM, David Hrb?c( wrote:

    Farkas Levente napsal(a):
    hi,
    as it was told that centos can be reproduced and all rpm build in
    mock i
    just made a quick test and try to rebuild all centos-5.3 srpms. the
    result not too bad but far from perfect. the setup was a fresh i386
    centos-5.3 minimal install + yum update + install
    mock-0.6.13-1.el5_2.3
    (the result was almost the same even worse with 2 srpms with
    mock-0.9.14-1.el5).
    from 1186 srpms
    Hi Farkas,
    what is you method to obtain rmp build list with correct dependencies?
    Could you share how you automate it?
    One way to automate is with repeated passes across a manifest of builds.

    Install successes, remove (or repeat that works too) from manifest,
    lather rinse repeat.

    That usually converges pretty fast, like maybe 5-10 passes.

    Ordering the manifest is often not too hard.

    Repeated traverses of a manifest has the distinct advantage that it does
    depend on the quality of build dependencies, which can be unreliable
    outside of an exactly identical build system process. Farkas was using
    mock iirc.

    73 de Jeff
  • David Hrbáč at Apr 24, 2009 at 2:57 pm

    Jeff Johnson napsal(a):
    One way to automate is with repeated passes across a manifest of builds.

    Install successes, remove (or repeat that works too) from manifest,
    lather rinse repeat.

    That usually converges pretty fast, like maybe 5-10 passes.

    Ordering the manifest is often not too hard.

    Repeated traverses of a manifest has the distinct advantage that it does
    depend on the quality of build dependencies, which can be unreliable
    outside of an exactly identical build system process. Farkas was using
    mock iirc.
    Jeff,
    well, so once again. I'm not asking about the building. I'm just curious
    what is Farka's method to prepare SRPMS list to build with respect to
    dependencies.
    David
  • Jeff Johnson at Apr 24, 2009 at 3:03 pm

    On Apr 24, 2009, at 10:57 AM, David Hrb?c( wrote:
    Jeff,
    well, so once again. I'm not asking about the building. I'm just
    curious
    what is Farka's method to prepare SRPMS list to build with respect to
    dependencies.
    And the answer was "Use mock." (which is what Farkas is using iirc).
    Mock will populate the build system using build dependencies, no
    "SRPMS list" involved.

    73 de Jeff
  • David Hrbáč at Apr 24, 2009 at 3:15 pm

    Jeff Johnson napsal(a):
    And the answer was "Use mock." (which is what Farkas is using iirc).
    Mock will populate the build system using build dependencies, no
    "SRPMS list" involved.
    We do not understand each other. Have 3 SRPMS we want to build, how do
    we know the order? Quite easy to resolve. Have 3000 SRPMS...
    David
  • David Hrbáč at Apr 24, 2009 at 3:24 pm

    David Hrb?c napsal(a):
    Jeff Johnson napsal(a):
    And the answer was "Use mock." (which is what Farkas is using iirc).
    Mock will populate the build system using build dependencies, no
    "SRPMS list" involved.
    Jeff,
    sorry, now I understand. Man mock :o).
    Thanks,
    David
  • Jeff Johnson at Apr 24, 2009 at 3:26 pm

    On Apr 24, 2009, at 11:15 AM, David Hrb?c( wrote:

    Jeff Johnson napsal(a):
    And the answer was "Use mock." (which is what Farkas is using iirc).
    Mock will populate the build system using build dependencies, no
    "SRPMS list" involved.
    We do not understand each other. Have 3 SRPMS we want to build, how do
    we know the order? Quite easy to resolve. Have 3000 SRPMS...
    Build dependencies tell the immediately adjacent prerequisites for
    building a package. Install those immediate prerequsites (and
    all of the other install prerequisites needed by the adjacent vertexes
    in the build dependency graph).

    That's the algorithm for determining how to build any
    single package.

    I don't know of anyone who has attempted to efficiently
    order all builds, not just a single build. There's almost
    loops involved, which need to be remedied by using an older
    version of some prerequiste in order to build the newer version.

    The process I outlined, with multiple passes across a list
    of SRPMS, keeping track of the order of successes, will
    generate the list you want for the next time you want
    to build 3000 SRPMS.

    73 de Jeff
  • Farkas Levente at Apr 24, 2009 at 9:55 pm

    David Hrb?c wrote:
    Farkas Levente napsal(a):
    hi,
    as it was told that centos can be reproduced and all rpm build in mock i
    just made a quick test and try to rebuild all centos-5.3 srpms. the
    result not too bad but far from perfect. the setup was a fresh i386
    centos-5.3 minimal install + yum update + install mock-0.6.13-1.el5_2.3
    (the result was almost the same even worse with 2 srpms with
    mock-0.9.14-1.el5).
    from 1186 srpms
    Hi Farkas,
    what is you method to obtain rmp build list with correct dependencies?
    Could you share how you automate it?
    how cares? no one. the real question what centos and redhat use to make
    a reproducible, working and reliable distro?
    the answer this is not solved:-(
    fyi: you're not even able to rebuild rhel/centos 5.3 on itself!
    this comes from the simple fact how these distros put together. let's
    start from 5.0 copy a few packages from fedora core 6 (!) both the srpms
    and the binaries, build a few packages on rhel4 rebuild them on the new
    system and if it's working together (ie. can run) then it's ok.
    then in 5.1 copy packages from 5.0, plus rebuild new packages. and the
    same for all new .x updates.
    one of the answer for my bugzilla report was:
    "to be able to rebuild the src.rpm is not our (redhat) goal":-) so
    ticket was closed.
    just ot mention a few problem with 5.3:
    - openjava was added to the distro so all packages which requires
    java-devel now try to build with openjava in stead of gcc's java and
    most of them fail.
    - new updates like dbus-glib, ifd-gate, pccs etc have incompatible devel
    packages eg. headers, but not all of the packages requires these new
    packages was rebuild/fixed so those packages no longer build.
    - newer gcc, toolchain etc (which included in later updates) have
    stronger check and standard compliance but with these tools old and
    buggy code no longer compile.
    - rpm and yum (and its' tools) far from perfect... suppose i don't use
    mock. how can i install all required package to buld X.src.rpm?
    apt-get build-dep is many years old, but with rpm?
    - rpmlint is almost unusable (it's warning about space and tabs:-), but
    no one really checks rhel's and fedora's spec file. eg. see mysql (a not
    so important package:-) it's build check is not in the %check section
    but in the %build. it's a serious bug? not we can put everything in the
    %prep and it's still working, but currently you can't rebuild mysql even
    on it's original rhel 5.2:-(

    imho before all release it must be a massrebuild of the whole distro in
    itself there is not any other solution.

    i was a redhat mirror site admin since redhat 3.0 (not rhel:-) and use
    redhat/centos/fedora since then but after 15 years i start to thing
    about ubuntu:-(

    --
    Levente "Si vis pacem para bellum!"
  • James Antill at Apr 27, 2009 at 2:56 pm

    On Fri, 2009-04-24 at 23:55 +0200, Farkas Levente wrote:

    just ot mention a few problem with 5.3:
    - openjava was added to the distro so all packages which requires
    java-devel now try to build with openjava in stead of gcc's java and
    most of them fail.
    - new updates like dbus-glib, ifd-gate, pccs etc have incompatible devel
    packages eg. headers, but not all of the packages requires these new
    packages was rebuild/fixed so those packages no longer build.
    - newer gcc, toolchain etc (which included in later updates) have
    stronger check and standard compliance but with these tools old and
    buggy code no longer compile.
    This is useless churn to rebuild all the packages to fix these kinds of
    build differences, why do you think RH's customers would want them to do
    that?
    - rpm and yum (and its' tools) far from perfect... suppose i don't use
    mock. how can i install all required package to buld X.src.rpm?
    apt-get build-dep is many years old, but with rpm?
    I'm not sure what you are trying to ask/say here. Maybe you want to
    know about yum-builddep in yum-utils? Why would you not want to use
    mock?
    Obviously rpm doesn't contain anything to do this, just as dpkg
    doesn't. Also AFAIK apt still kind of works in RHEL-5, although I still
    wouldn't recommend it.
    - rpmlint is almost unusable (it's warning about space and tabs:-), but
    no one really checks rhel's and fedora's spec file. eg. see mysql (a not
    so important package:-) it's build check is not in the %check section
    but in the %build. it's a serious bug? not we can put everything in the
    %prep and it's still working, but currently you can't rebuild mysql even
    on it's original rhel 5.2:-(
    This is something that's done at the Fedora level, for obvious (I'd
    assume) reasons, you don't want to be changing specfiles just to comply
    with rpmlint in something like RHEL.

    --
    James Antill <james at fedoraproject.org>
    Fedora
  • Farkas Levente at Apr 27, 2009 at 3:05 pm

    James Antill wrote:
    On Fri, 2009-04-24 at 23:55 +0200, Farkas Levente wrote:

    just ot mention a few problem with 5.3:
    - openjava was added to the distro so all packages which requires
    java-devel now try to build with openjava in stead of gcc's java and
    most of them fail.
    - new updates like dbus-glib, ifd-gate, pccs etc have incompatible devel
    packages eg. headers, but not all of the packages requires these new
    packages was rebuild/fixed so those packages no longer build.
    - newer gcc, toolchain etc (which included in later updates) have
    stronger check and standard compliance but with these tools old and
    buggy code no longer compile.
    This is useless churn to rebuild all the packages to fix these kinds of
    build differences, why do you think RH's customers would want them to do
    that?
    simple because in the current system the current src.rpms are not able
    to build. so my system source can't be rebuild from my system src.rpm:-(
    - rpm and yum (and its' tools) far from perfect... suppose i don't use
    mock. how can i install all required package to buld X.src.rpm?
    apt-get build-dep is many years old, but with rpm?
    I'm not sure what you are trying to ask/say here. Maybe you want to
    know about yum-builddep in yum-utils? Why would you not want to use
    mock?
    ask:
    how can i install all required package to buld X.src.rpm with yum/rpm?
    - rpmlint is almost unusable (it's warning about space and tabs:-), but
    no one really checks rhel's and fedora's spec file. eg. see mysql (a not
    so important package:-) it's build check is not in the %check section
    but in the %build. it's a serious bug? not we can put everything in the
    %prep and it's still working, but currently you can't rebuild mysql even
    on it's original rhel 5.2:-(
    This is something that's done at the Fedora level, for obvious (I'd
    assume) reasons, you don't want to be changing specfiles just to comply
    with rpmlint in something like RHEL.
    what i try to say currently there is no good spec file checking tool and
    there not forced any check on the used specfile. imho it'd be useful if
    all package would be checked before put into production, but there is no
    such thing currently.



    --
    Levente "Si vis pacem para bellum!"
  • Jim Perrin at Apr 27, 2009 at 3:14 pm

    On Mon, Apr 27, 2009 at 11:05 AM, Farkas Levente wrote:

    ?I'm not sure what you are trying to ask/say here. Maybe you want to
    know about yum-builddep in yum-utils? Why would you not want to use
    mock?
    ask:
    how can i install all required package to buld X.src.rpm with yum/rpm?
    This is basically exactly what mock does for you. It uses the
    buildrequires listing in the spec of the src.rpm to install the
    dependencies. There are some src.rpms that may not list all the
    required packages, but they aren't that common.
    what i try to say currently there is no good spec file checking tool and
    there not forced any check on the used specfile. imho it'd be useful if
    all package would be checked before put into production, but there is no
    such thing currently.
    How is this a centos issue? We have to deal with the src.rpms from
    upstream in the same manner. If bugs are being inserted by centos
    that's one thing. If they're the same as the upstream bugs, I don't
    see a problem. Everything should be compatible, including the flaws.


    --
    During times of universal deceit, telling the truth becomes a revolutionary act.
    George Orwell
  • Jeff Johnson at Apr 27, 2009 at 3:09 pm

    On Apr 27, 2009, at 10:56 AM, James Antill wrote:
    Obviously rpm doesn't contain anything to do this, just as dpkg
    doesn't. Also AFAIK apt still kind of works in RHEL-5, although I
    still
    wouldn't recommend it.
    Your analogy as well as your knowledge is naive ...

    But hey "Use yum!"

    73 de Jeff
  • Les Mikesell at Apr 27, 2009 at 5:44 pm

    James Antill wrote:
    just ot mention a few problem with 5.3:
    - openjava was added to the distro so all packages which requires
    java-devel now try to build with openjava in stead of gcc's java and
    most of them fail.
    - new updates like dbus-glib, ifd-gate, pccs etc have incompatible devel
    packages eg. headers, but not all of the packages requires these new
    packages was rebuild/fixed so those packages no longer build.
    - newer gcc, toolchain etc (which included in later updates) have
    stronger check and standard compliance but with these tools old and
    buggy code no longer compile.
    This is useless churn to rebuild all the packages to fix these kinds of
    build differences, why do you think RH's customers would want them to do
    that?
    I thought _THE_ selling point of open source has always been that in
    case of problems the vendor can't/won't fix, you have the option to make
    the change yourself. But if you can't rebuild their packages or even
    tell how the source relates to the shipped binary, that isn't true and
    shouldn't be represented as such.

    --
    Les Mikesell
    lesmikesell at gmail.com
  • Mike A. Harris at Apr 28, 2009 at 8:37 am

    Les Mikesell wrote:
    James Antill wrote:
    just ot mention a few problem with 5.3:
    - openjava was added to the distro so all packages which requires
    java-devel now try to build with openjava in stead of gcc's java and
    most of them fail.
    - new updates like dbus-glib, ifd-gate, pccs etc have incompatible devel
    packages eg. headers, but not all of the packages requires these new
    packages was rebuild/fixed so those packages no longer build.
    - newer gcc, toolchain etc (which included in later updates) have
    stronger check and standard compliance but with these tools old and
    buggy code no longer compile.
    This is useless churn to rebuild all the packages to fix these kinds of
    build differences, why do you think RH's customers would want them to do
    that?
    I thought _THE_ selling point of open source has always been that in
    case of problems the vendor can't/won't fix, you have the option to make
    the change yourself. But if you can't rebuild their packages or even
    tell how the source relates to the shipped binary, that isn't true and
    shouldn't be represented as such.
    It sucks that the latest sources do not compile on the OS they were
    originally built for, but the fact is that packages get built at a
    certain point in time against what is in the tree at that time.

    Then packages continue to be updated to fix bugs, etc. and in some cases
    maybe even add a new feature here or there. All it takes is for one of
    these updates to change something in such a manner that packages that
    depend on this package will no longer compile. It could be as simple as
    a file being moved from one location to another, or some other innocent
    innocuous change.

    The only way to prevent that sort of thing is to have a mandatory policy
    that whenever _any_ package is rebuilt in the distribution, that the
    entire distribution must be rebuilt in a tree with the absolute latest
    packages present just to ensure that all packages still compile.

    What if they don't? Are they going to be expected to rebuild all
    packages that don't build because of this, and have to release them all
    as updates? That makes no sense whatsoever _except_ during the initial
    development phase of the OS - ie: rawhide.

    Could the whole process be improved? Absolutely. Is it? I think so.
    Just look at how Fedora policies and procedures have evolved since
    inception. Look at how the build system has evolved. Keep in mind that
    RHEL inherits from Fedora initially and so it also inherits all of the
    improvements done in this area.

    As things stand right now though, the only way for anyone to
    successfully build any package that shipped with RHEL and have it
    successfully build, is to know the exact version of every package that
    was installed on the system that built the RHEL package successfully. I
    don't know if that information is logged and/or kept or not but I
    suspect not.

    So the distro is built against itself during development, but it isn't
    kept in a rolling state of 100% self-hostingness. It would be nice if
    the latest packages always built against the current OS, but the cost of
    making it do that could be quite high, and easily result in tonnes of
    packages being issued as updates for no other reason than to make them
    compile again.

    It's really just the nature of how everything builds together. There
    are no conspiracies going on. This problem will IMHO become less and
    less of an issue in the future as the buildsystem and policies continue
    to mature and evolve on the Fedora/Red Hat side of things.
  • James Olin Oden at Apr 28, 2009 at 5:07 pm

    On Tue, Apr 28, 2009 at 4:37 AM, Mike A. Harris wrote:
    Les Mikesell wrote:
    James Antill wrote:
    just ot mention a few problem with 5.3:
    - openjava was added to the distro so all packages which requires
    java-devel now try to build with openjava in stead of gcc's java and
    most of them fail.
    - new updates like dbus-glib, ifd-gate, pccs etc have incompatible devel
    packages eg. headers, but not all of the packages requires these new
    packages was rebuild/fixed so those packages no longer build.
    - newer gcc, toolchain etc (which included in later updates) have
    stronger check and standard compliance but with these tools old and
    buggy code no longer compile.
    ?This is useless churn to rebuild all the packages to fix these kinds of
    build differences, why do you think RH's customers would want them to do
    that?
    I thought _THE_ selling point of open source has always been that in
    case of problems the vendor can't/won't fix, you have the option to make
    the change yourself. ?But if you can't rebuild their packages or even
    tell how the source relates to the shipped binary, that isn't true and
    shouldn't be represented as such.
    It sucks that the latest sources do not compile on the OS they were
    originally built for, but the fact is that packages get built at a
    certain point in time against what is in the tree at that time.
    <snip>
    Your spending a lot of time defending the status quo, but your not
    really answering an honest complaint. I've spent a lot of time
    servicing customers, and when they complain to you, they really don't
    care about explanations that don't try to help them with their
    immediate problem. Typically, at the point they start hearing long
    explanations of why things are the way they are without movement in
    the direction of helping them with their particular problems they just
    move on.

    I think the salient points are:

    * RedHat distros today don't self host.
    * Some customers desire that the distro could self host.

    Maybe the number of customers that desire this are too small to worry
    about, versus the cost of meeting their perceived needs. Maybe,
    solving the problem is really hard. None of that changes the above
    two facts; it only places them in a context.

    Cheers...james
  • Les Mikesell at Apr 28, 2009 at 5:28 pm

    James Olin Oden wrote:
    I think the salient points are:

    * RedHat distros today don't self host.
    * Some customers desire that the distro could self host.

    Maybe the number of customers that desire this are too small to worry
    about, versus the cost of meeting their perceived needs. Maybe,
    solving the problem is really hard. None of that changes the above
    two facts; it only places them in a context.
    Do you know how the other distributions handle this? Obviously, gentoo
    was designed with end-user source rebuilds in mind and the *bsd's got it
    right eons ago, but what about debian, ubuntu, etc.

    --
    Les Mikesell
    lesmikesell at gmail.com
  • James Olin Oden at Apr 28, 2009 at 5:32 pm

    On Tue, Apr 28, 2009 at 1:28 PM, Les Mikesell wrote:
    James Olin Oden wrote:
    I think the salient points are:

    ? ?* RedHat distros today don't self host.
    ? ?* Some customers desire that the distro could self host.

    Maybe the number of customers that desire this are too small to worry
    about, versus the cost of meeting their perceived needs. ?Maybe,
    solving the problem is really hard. ? None of that changes the above
    two facts; it only places them in a context.
    Do you know how the other distributions handle this? ?Obviously, gentoo
    was designed with end-user source rebuilds in mind and the *bsd's got it
    right eons ago, but what about debian, ubuntu, etc.
    I've no idea how others do this. I would likely have tools in place
    to do source annalysis to figure out when most things need to be
    rebuilt. Also, after each package is built I would probably have a
    build all happen just make sure things still build. Only need the one
    package, unless there is a problem with the build all. Probably,
    this view is too simplistic though. Sadly, I only know that
    self-hosting is important to me; I don't know how to get there.

    ...james
  • Dag Wieers at Apr 28, 2009 at 6:19 pm

    On Tue, 28 Apr 2009, James Olin Oden wrote:

    Your spending a lot of time defending the status quo, but your not
    really answering an honest complaint.
    What you see as a defense, I saw as an explanation to why it is the way it
    is. I think nobody argues that it is great the way it is. Especially
    within the CentOS project because it hurts our effort too.

    I've spent a lot of time servicing customers, and when they complain to
    you, they really don't care about explanations that don't try to help
    them with their immediate problem.
    I am not disagreeing with you at all, but I don't see any solution from
    you either.

    CentOS does not have customers, and nothing should hold you back to
    implement a solution and offer it to Red Hat. CentOS can even help
    you with the resources if you want to make a CentOS project out of it.
    (Although it probably makes more sense as a Fedora or Red Hat project)

    But at the moment this is yet another task that requires people's time
    and I am not sure if we want to dedicate existing CentOS people to such a
    project if it has an impact on more important deliverables.

    The question really is: are you willing to be part of the solution ?

    I'd say we need more people to take action, and less people to argue :)

    If nobody stands up to fix this, we might see it as a sign that nobody
    thinks it is important (enough).

    Kind regards,
    --
    -- dag wieers, dag at centos.org, http://dag.wieers.com/ --
    [Any errors in spelling, tact or fact are transmission errors]
  • James Olin Oden at Apr 28, 2009 at 7:33 pm

    On Tue, Apr 28, 2009 at 2:19 PM, Dag Wieers wrote:
    On Tue, 28 Apr 2009, James Olin Oden wrote:

    Your spending a lot of time defending the status quo, but your not
    really answering an honest complaint.
    What you see as a defense, I saw as an explanation to why it is the way it
    is. I think nobody argues that it is great the way it is. Especially
    within the CentOS project because it hurts our effort too.

    I've spent a lot of time servicing customers, and when they complain to
    you, they really don't care about explanations that don't try to help
    them with their immediate problem.
    I am not disagreeing with you at all, but I don't see any solution from
    you either.
    I wasn't offering one. I was more defending the ones that were
    dismissed out of hand for what actually is a reasonable desire (i.e.
    that either RedHat was self hostable, or that the method for building
    it was clearly documented).
    CentOS does not have customers,
    Yes you do. Not paying, but you have those you serve. Now you serve
    them at your pleasure, but serve them you do (and quite well I might
    add). My comments though were not directed at CentOS. You literally
    inherit what RedHat gives you. You can make it better but typically
    your goals seem to be to stay as close to the original as possible
    (which is a good goal).
    and nothing should hold you back to
    implement a solution and offer it to Red Hat.
    Lots of things would hold me back and they are the same things that
    hold you back.
    CentOS can even help
    you with the resources if you want to make a CentOS project out of it.
    (Although it probably makes more sense as a Fedora or Red Hat project) Indeed.
    But at the moment this is yet another task that requires people's time
    and I am not sure if we want to dedicate existing CentOS people to such a
    project if it has an impact on more important deliverables.
    I would agree.
    The question really is: are you willing to be part of the solution ?
    No that was not the question. The question was, "How does one
    reproducibly build a RedHat distribution?". A very reasonable
    question to ask.
    I'd say we need more people to take action, and less people to argue :)
    Sure...wasn't trying to argue, was more trying to defend. Also,
    when/if I find something that my employer really needs fixed I fix it
    and send patches upstream. I just don't think this is a technical
    problem.
    If nobody stands up to fix this, we might see it as a sign that nobody
    thinks it is important (enough).
    Non-redhat employees have no direct way of fixing what is not a
    technical problem but a process problem. The only way that they have
    of effecting a process problem is let RedHat know that its a problem
    for them. RedHat may or may not then decide to do anything about it.
    They may choose for what they feel are very valid business reasons,
    to simply maintain the status quo. If more customers communicate
    their displeasure, in a reasonable manner, regarding an issue, this
    may give more weight to it. After all they can't just do everything
    they are requested by everyone. Or again, the cost to fix their
    process may be so prohibitive they may choose to not do anything.

    Even though this hypothetical response is reasonable, the customer may
    see that the issue is such a big deal that they just need to move on,
    provided of course there is somewhere to move on to. This is a
    reasonable response of the customer.

    This is really the heart of what I was trying to say. It is
    _reasonable_ for customers of RedHat, be they majority or minority, to
    take issue with RedHat's build process if the result produces
    something that is undesirable to them. My entire purpose in this
    case was to defend the reasonableness of this type inquiry.

    As it is, we both agree that the situation is not desirable.

    Cheers...james

    P.S. Now I did bring up a premise that may not be true, which was that
    the issue is not "Technical" but that it was a RedHat "Process"
    problem. If I'm wrong in that regard, then actually a solution is
    achievable, and one might work on the problem, and not be a RedHat
    employee. I do agree that there are technical things one could do to
    help solve the problem, but its seems to me that the main thing they
    need is an automated rebuild of the world to detect if there is indeed
    a problem with the build being self hostable. I would guess RedHat
    has this capability in their build system, and if they do, it would be
    a decision to use it or not...which is a process issue.
  • Seth Vidal at Apr 28, 2009 at 8:38 pm

    On Tue, 28 Apr 2009, James Olin Oden wrote:

    Cheers...james

    P.S. Now I did bring up a premise that may not be true, which was that
    the issue is not "Technical" but that it was a RedHat "Process"
    problem. If I'm wrong in that regard, then actually a solution is
    achievable, and one might work on the problem, and not be a RedHat
    employee. I do agree that there are technical things one could do to
    help solve the problem, but its seems to me that the main thing they
    need is an automated rebuild of the world to detect if there is indeed
    a problem with the build being self hostable. I would guess RedHat
    has this capability in their build system, and if they do, it would be
    a decision to use it or not...which is a process issue.

    I'm not going to comment on red hat processes b/c I honestly don't know
    them and cannot answer for them. I do know a few things about
    Fedora's processes for completely rebuilding the distro and since some of
    the future rhel processes come out of the fedora processes I think looking
    at how a complete rebuild of fedora happens might be the most valuable.

    Normally, when fedora is being rebuilt we build on whatever has come
    before and what's available and we do it all in mock in build dep order.

    Now, those deps are better in fedora 7,8,9,10 and rawhide than it was in
    fedora core 6 (what rhel5 was based off of).

    We install a publically published set of base pkgs and that's what the
    build environment starts with.

    then we rebuild the srpm inside the mock chroot from the spec we've made

    then we build the package.

    The resulting packages are added to the set of repositories that mock can
    pull from for future builds.

    Then the chroot is cleaned and reinstalled and the build begins for the
    next pkg in the chain.

    Does that help?

    -sv
  • James Olin Oden at Apr 28, 2009 at 8:44 pm

    On Tue, Apr 28, 2009 at 4:38 PM, Seth Vidal wrote:
    On Tue, 28 Apr 2009, James Olin Oden wrote:

    Cheers...james

    P.S. Now I did bring up a premise that may not be true, which was that
    the issue is not "Technical" but that it was a RedHat "Process"
    problem. ?If I'm wrong in that regard, then actually a solution is
    achievable, and one might work on the problem, and not be a RedHat
    employee. ?I do agree that there are technical things one could do to
    help solve the problem, but its seems to me that the main thing they
    need is an automated rebuild of the world to detect if there is indeed
    a problem with the build being self hostable. ? I would guess RedHat
    has this capability in their build system, and if they do, it would be
    a decision to use it or not...which is a process issue.

    I'm not going to comment on red hat processes b/c I honestly don't know
    them and cannot answer for them. I do know a few things about
    Fedora's processes for completely rebuilding the distro and since some of
    the future rhel processes come out of the fedora processes I think looking
    at how a complete rebuild of fedora happens might be the most valuable.

    Normally, when fedora is being rebuilt we build on whatever has come
    before and what's available and we do it all in mock in build dep order.

    Now, those deps are better in fedora 7,8,9,10 and rawhide than it was in
    fedora core 6 (what rhel5 was based off of).

    We install a publically published set of base pkgs and that's what the
    build environment starts with.

    then we rebuild the srpm ?inside the mock chroot from the spec we've made

    then we build the package.

    The resulting packages are added to the set of repositories that mock can
    pull from for future builds.

    Then the chroot is cleaned and reinstalled and the build begins for the
    next pkg in the chain.

    Does that help?
    Yes.

    Now along this sortid trail of email messages, there was mention that
    some packages no longer build with the current compilers. Is that
    not the case, or was that a mis-statement?

    Because, it sounds Fedora at this point is self-hosting by that
    description, and if RedHat follows that practice it too will be self
    hosting.

    ...james
  • Seth Vidal at Apr 28, 2009 at 8:50 pm

    On Tue, 28 Apr 2009, James Olin Oden wrote:

    Yes.

    Now along this sortid trail of email messages, there was mention that
    some packages no longer build with the current compilers. Is that
    not the case, or was that a mis-statement?

    Because, it sounds Fedora at this point is self-hosting by that
    description, and if RedHat follows that practice it too will be self
    hosting.
    I do not know for rhel5. I'm not involved in building rhel5 so I can't
    tell you one way or the other about that.

    I know that we completed a bump and rebuild in fedora 11/rawhide as of feb
    25 2009. https://fedoraproject.org/wiki/Fedora_11_Mass_Rebuild

    And unless I'm missing something it is correct that fedora 11/rawhide is
    self hosting.

    All of the processes and programs that fedora uses to build its distro are
    public and available. There are a number of secondary architectures now
    which are rebuilding from scratch. Two that come to mind are arm and
    sparc.

    I hope this helps. I'm sorry I can't tell you much about rhel, I just
    don't have any firsthand experience in how the internal build process
    works.

    -sv
  • Mike A. Harris at Apr 30, 2009 at 10:08 am

    James Olin Oden wrote:

    Now along this sortid trail of email messages, there was mention that
    some packages no longer build with the current compilers. Is that
    not the case, or was that a mis-statement?
    If you peruse the Fedora ftp repository (or via http) you'll likely find
    some packages present with the .fcN dist tag in them where N < current
    release of Fedora. That was true for Fedora 8, and I believe Fedora 10
    as well. Not sure if it is true for Fedora 11/rawhide currently or not
    however.
    Because, it sounds Fedora at this point is self-hosting by that
    description, and if RedHat follows that practice it too will be self
    hosting.
    It is a good process for building clean rpms, at least cleaner than
    prior methods. It can not guarantee self hosting however because once
    any rpm is built - if any of its dependencies are rebuilt, if the
    compiler or toolchain changes, etc. - it is possible any of those
    changes could cause any number of rpms previously built to fail. As
    such, in order to guarantee self hosting, if say... gcc were rebuilt,
    then the entire repository would need to be rebuilt, and as any rpms
    were built - anything that used them as dependencies would need to be
    rebuilt again. There are packages that have build loops, so they might
    need to be rebuilt in a loop a few times to ensure they still build.

    I'm not saying this shouldn't be done, but it is a large amount of
    potential package churn to say the least. It would be nice however if
    mass rebuilds were done more often during the OS development phase, and
    failures were mandatorily fixed. That would lead closer towards self
    hosting becoming more of a reality.

    I don't think the process should be continued post-final though as it
    would churn excessive unnecessary updates to end users.
  • Farkas Levente at Apr 30, 2009 at 10:45 am

    Mike A. Harris wrote:
    It is a good process for building clean rpms, at least cleaner than
    prior methods. It can not guarantee self hosting however because once
    any rpm is built - if any of its dependencies are rebuilt, if the
    compiler or toolchain changes, etc. - it is possible any of those
    changes could cause any number of rpms previously built to fail. As
    such, in order to guarantee self hosting, if say... gcc were rebuilt,
    then the entire repository would need to be rebuilt, and as any rpms
    were built - anything that used them as dependencies would need to be
    rebuilt again. There are packages that have build loops, so they might
    need to be rebuilt in a loop a few times to ensure they still build.
    but this rebuild at least can check which packages not build and for
    those which build don't do anything. anyway the best would be if we can
    rebuild all dependencies and some kind of 'smart diff' can compare it
    with previous build and push updates only if 'smart diff' differ.

    --
    Levente "Si vis pacem para bellum!"
  • Bogdan Costescu at Apr 30, 2009 at 12:19 pm

    On Thu, 30 Apr 2009, Mike A. Harris wrote:

    I don't think the process should be continued post-final though as
    it would churn excessive unnecessary updates to end users.
    ... and would make QA impossible. Red Hat does its QA on the binary
    packages that they provide and if they are happy with the result it
    doesn't matter if the package was rebuilt the day before or came out
    of a Fedora build one year ago. The whole idea of using well known
    binaries (and which go through QA and can be supported) is what keeps
    people paying to Red Hat. It's also what keeps many people downloading
    CentOS packages and not rebuilding the SRPMs themselves. If there's a
    problem they can compare with others running _the same binaries_ and
    find out if the problem comes from the binary or not; with binary
    packages built in a random order and random build environment, there
    is no possibility to talk about reproducible behaviour and makes any
    comparison meaningless.

    So how do you (the OP) propose to do QA and bug tracking when each
    single package update can trigger an avalanche of other updates ?

    Please note that I also find good the idea of self-hosting. It's a
    very good feeling to take the SRPM and have a binary package after
    just a 'rpmbuild -ba'. But I think that this goal has to be balanced
    with other goals to make a distribution.

    --
    Bogdan Costescu

    IWR, University of Heidelberg, INF 368, D-69120 Heidelberg, Germany
    Phone: +49 6221 54 8240, Fax: +49 6221 54 8850
    E-mail: bogdan.costescu at iwr.uni-heidelberg.de
  • Farkas Levente at Apr 28, 2009 at 9:19 pm

    Seth Vidal wrote:
    On Tue, 28 Apr 2009, James Olin Oden wrote:

    Cheers...james

    P.S. Now I did bring up a premise that may not be true, which was that
    the issue is not "Technical" but that it was a RedHat "Process"
    problem. If I'm wrong in that regard, then actually a solution is
    achievable, and one might work on the problem, and not be a RedHat
    employee. I do agree that there are technical things one could do to
    help solve the problem, but its seems to me that the main thing they
    need is an automated rebuild of the world to detect if there is indeed
    a problem with the build being self hostable. I would guess RedHat
    has this capability in their build system, and if they do, it would be
    a decision to use it or not...which is a process issue.

    I'm not going to comment on red hat processes b/c I honestly don't know
    them and cannot answer for them. I do know a few things about
    Fedora's processes for completely rebuilding the distro and since some of
    the future rhel processes come out of the fedora processes I think looking
    at how a complete rebuild of fedora happens might be the most valuable.
    the biggest problem it's not enough. ie. it's not enough to sometime do
    a massrebuild (it's just happened because of gcc and rpm update), but
    under normal circumstances we can't do a massrebuild for all new
    packages (until we'll have so much computational power:-)
    imho the current fedora process is also bad (ie. the same can happened
    in fedora like in rhel with ifd-egate and pcsc-lite).

    ps. i think i start a new thread on fedora-devel since it's nothing to
    do with centos since the centos team can't do anything with upstream
    problems.

    --
    Levente "Si vis pacem para bellum!"
  • Dag Wieers at Apr 28, 2009 at 9:32 pm

    On Tue, 28 Apr 2009, James Olin Oden wrote:
    On Tue, Apr 28, 2009 at 2:19 PM, Dag Wieers wrote:
    On Tue, 28 Apr 2009, James Olin Oden wrote:

    CentOS does not have customers,
    Yes you do. Not paying, but you have those you serve. Now you serve
    them at your pleasure, but serve them you do (and quite well I might
    add). My comments though were not directed at CentOS. You literally
    inherit what RedHat gives you. You can make it better but typically
    your goals seem to be to stay as close to the original as possible
    (which is a good goal).
    I wouldn't call it customers. And we don't have employees either. :)

    We have users, and some of them are volunteers and that is the first thing
    to understand a project like CentOS. What is driving the project is a
    combination of self-interest and recognition of those volunteers.

    Any user can become a volunteer given some rules and some skills. Although
    currently that does not always scale that well and we are looking
    for ways to improve this and empower people to take action too.

    As a project we need to regularly do a self-assessement and better define
    who we are, what our goals are and how we want to reach those. And
    communicate this clearly to get all heads in the same direction.

    If you look at the past year we have made a lot of progress in some
    areas but we fell short on other domains. And I think we have seen
    the limits of working with volunteers a few times too.

    We have to learn from what we did wrong, and find spots to improve, but
    most of all we have to manage the resources we have better so that we can
    put trust in the people that dedicate their time for free and plan ahead.

    Anyway I am getting off-topic now.

    Customers seems to imply we have an incentive/obligation to support/help,
    and I am afraid we do not. Neither the project nor the community has any
    obligations, nevertheless people are being helped. We have users and we
    are users.

    --
    -- dag wieers, dag at centos.org, http://dag.wieers.com/ --
    [Any errors in spelling, tact or fact are transmission errors]
  • William L. Maltby at Apr 28, 2009 at 10:10 pm

    On Tue, 2009-04-28 at 23:32 +0200, Dag Wieers wrote:
    On Tue, 28 Apr 2009, James Olin Oden wrote:
    On Tue, Apr 28, 2009 at 2:19 PM, Dag Wieers wrote:
    On Tue, 28 Apr 2009, James Olin Oden wrote:

    CentOS does not have customers,
    Yes you do. <snip>
    I wouldn't call it customers. And we don't have employees either. :)

    We have users, and some of them are volunteers and that is the first thing
    to understand a project like CentOS. What is driving the project is a
    combination of self-interest and recognition of those volunteers.

    Any user can become a volunteer given some rules and some skills. Although
    currently that does not always scale that well and we are looking
    for ways to improve this and empower people to take action too.

    As a project we need to regularly do a self-assessement and better define
    who we are, what our goals are and how we want to reach those. And
    communicate this clearly to get all heads in the same direction.

    If you look at the past year we have made a lot of progress in some
    areas but we fell short on other domains. And I think we have seen
    the limits of working with volunteers a few times too.

    We have to learn from what we did wrong, and find spots to improve, but
    most of all we have to manage the resources we have better so that we can
    put trust in the people that dedicate their time for free and plan ahead.

    Anyway I am getting off-topic now.

    Customers seems to imply we have an incentive/obligation to support/help,
    and I am afraid we do not. Neither the project nor the community has any
    obligations, nevertheless people are being helped. We have users and we
    are users.
    I disagree about obligations. The obligations are self-imposed and, I
    would presume, are taken seriously by those who undertake those
    obligations. Without this there would be no project. Each project member
    must be able to have a certain reliance on the other project members, to
    some unspecified-here degree, to both carry out the tasks they agree to
    and do so in a "best effort" manner, allowing for many other factors.

    Without this degree of reliance, enabled by self-imposed obligation, the
    fabric of the project unravels - maybe quickly maybe over a long time.

    Once one self-imposes this obligation, and states it so that others may
    rely on that, it becomes more than "self-imposed" until such time as the
    obligation is either publicly (within the project community) withdrawn
    or the consistent long-term failure to satisfy the obligation is noted,
    at which time other members must cease reliance on that obligation.

    Now, unless one is a totally amoral individual, this obligation also
    extends to the users, whether explicitly stated or not. Without the user
    community accepting that you (the project) have undertaken an
    obligation, they can not rely upon the project and should seek another
    venue if that is an issue for their scenario.

    The side-effect of being able to rely upon you (the project) is a
    willingness of the/some users to return value, in various forms, to the
    project. It may be as participation in support of other user's queries,
    contributing to wiki contents, etc. or donation of $$, hardware, etc.

    That constitutes "payment", even if in kind. That could reasonably be
    argued to constitute a customer relationship.

    The fact that something other than money is or may be the form of
    payment does not alter the relationship.

    Many projects die a slow and lingering death because the project members
    and/or users are not willing to give up *any* "freedom" or make any
    concessions in recognition of this actual relationship.

    Unfortunately, the folks who *do* make such a commitment are often the
    ones most severely harmed by those who will not make such a commitment.
    It all has a relationship to a sense of "team" and "teamwork".
    From all this is derived another form of payment: a sense of
    satisfaction from a job well done, both individually and as a team. If
    you doubt such, take a look at the emotional and sometimes defensive
    responses when some outside the project post what is taken as an
    "unjust" criticism or an unreasonable and selfish request, without
    consideration for those who are doing the "heavy lifting".

    I could go on, but I hope you see the validity of this POV.

    >

    MHO
    --
    Bill
  • Jeff Johnson at Apr 28, 2009 at 10:17 pm

    On Apr 28, 2009, at 6:10 PM, William L. Maltby wrote:
    I could go on, but I hope you see the validity of this POV.
    It sounds lie a great plan, well reasoned, and rational,
    clearly listing the costs/benefits of creating and participating
    in a OSS project, and astutely pointing out that any
    project that doesn't understand the OSS model is eventually doomed.

    Its a shame that reality of OSS projects invariably comes to the
    boring conclusion
    We're all doomed.
    from (if nothing else) the 3 laws of thermodynamics.

    But the vendor/capitalist model for software has far deeper flaws
    than the OSS model you've described.

    73 de Jeff I was born an optimist!
  • Phil Schaffner at Apr 29, 2009 at 12:41 am
    On Tue, 2009-04-28 at 18:17 -0400, Jeff Johnson wrote:
    ...
    It sounds lie [sic] a great plan, well reasoned, and rational,
    clearly listing the costs/benefits of creating and participating
    in a OSS project, and astutely pointing out that any
    project that doesn't understand the OSS model is eventually doomed.

    Its a shame that reality of OSS projects invariably comes to the
    boring conclusion
    We're all doomed.
    from (if nothing else) the 3 laws of thermodynamics.
    Well, from this statement at least we will continue:

    Zeroth: You must play the game.
    First: You can't win.
    Second: You can't break even.
    Third: You can't quit the game.

    A more formal statement of the 2nd law may be applicable: The entropy of
    an isolated system which is not in equilibrium will tend to increase
    over time, approaching a maximum value at equilibrium.
    But the vendor/capitalist model for software has far deeper flaws
    than the OSS model you've described.
    So, paraphrasing Churchill, OSS is the worst form of software
    development except all the others that have been tried.
    73 de Jeff I was born an optimist!
    Ah, then hopefully humans can do better than thermodynamic systems!
    :-)

    Cheers,
    Phil
  • Karanbir Singh at Apr 28, 2009 at 11:15 pm

    James Olin Oden wrote:
    I think the salient points are:

    * RedHat distros today don't self host.
    If you are refering to CentOS-2.1 or CentOS-3 I'd agree. But on CentOS-4
    and CentOS-5 that is simply not true. I guess you are using a small
    window onto what really is a distro to mean a snapshot in time.

    As I said before, CentOS exists and that should prove that the distro is
    self hosting in its entireity. There are pkgs with a .fc6. tag - but
    look at their builddates and times. They were built completely within
    the distro host. No fedora6 was used to build anything in centos-5.

    --
    Karanbir Singh : http://www.karan.org/ : 2522219 at icq
  • James Olin Oden at Apr 29, 2009 at 12:09 am

    On Tue, Apr 28, 2009 at 7:15 PM, Karanbir Singh wrote:
    James Olin Oden wrote:
    I think the salient points are:

    ? ?* RedHat distros today don't self host.
    If you are refering to CentOS-2.1 or CentOS-3 I'd agree. But on CentOS-4
    and CentOS-5 that is simply not true. I guess you are using a small
    window onto what really is a distro to mean a snapshot in time.

    As I said before, CentOS exists and that should prove that the distro is
    self hosting in its entireity. There are pkgs with a .fc6. tag - but
    look at their builddates and times. They were built completely within
    the distro host. No fedora6 was used to build anything in centos-5.
    Please, kindly and graciously note that I was not the original poster
    to this conversation, but only joined after someone "confirmed" that
    RHEL 5 at the moment (not CentOS) was not self hosting by stating
    that though that is the unfortunate state of affairs, there are so
    many reasons why its this way. If its self hosting great. If its not
    I'm concerned...but that really really wasn't my point.

    ...james
  • Farkas Levente at Apr 30, 2009 at 10:51 am

    Karanbir Singh wrote:
    James Olin Oden wrote:
    I think the salient points are:

    * RedHat distros today don't self host.
    If you are refering to CentOS-2.1 or CentOS-3 I'd agree. But on CentOS-4
    and CentOS-5 that is simply not true. I guess you are using a small
    window onto what really is a distro to mean a snapshot in time.
    yes. eg. i call centos-5.3 'a distro' which has an install iso.
    As I said before, CentOS exists and that should prove that the distro is
    self hosting in its entireity. There are pkgs with a .fc6. tag - but
    look at their builddates and times. They were built completely within
    the distro host. No fedora6 was used to build anything in centos-5.
    as i start this thread i repeat myself again. this is nothing against
    centos itself and neither the fault of the centos team nor you. what's
    more i now start to understand why the 5.3 release takes so long and
    understand why it's requires so much manual work to build the system and
    start to respect the whole centos team much more.
    i simple state rhel/centos 5.3 can't be rebuild on itself without
    patching spec files and adding patches for a few files.
    imho about 10-20% can't compile from this 10-20% most can be build with
    small hacks and the remaining ~4% need extra work.


    --
    Levente "Si vis pacem para bellum!"
  • Ralph Angenendt at Apr 30, 2009 at 11:09 am

    Farkas Levente wrote:
    i simple state rhel/centos 5.3 can't be rebuild on itself without
    patching spec files and adding patches for a few files.
    And yet it is.

    Ralph
    -------------- next part --------------
    A non-text attachment was scrubbed...
    Name: not available
    Type: application/pgp-signature
    Size: 189 bytes
    Desc: not available
    Url : http://lists.centos.org/pipermail/centos-devel/attachments/20090430/103c1fd9/attachment.bin
  • Farkas Levente at Apr 30, 2009 at 11:16 am

    Ralph Angenendt wrote:
    Farkas Levente wrote:
    i simple state rhel/centos 5.3 can't be rebuild on itself without
    patching spec files and adding patches for a few files.
    And yet it is.
    did you try it?
    show me your build log the rebuild of ifd-egate-0.05-15.i386.rpm:-)

    --
    Levente "Si vis pacem para bellum!"
  • Ralph Angenendt at Apr 30, 2009 at 11:26 am

    Farkas Levente wrote:
    Ralph Angenendt wrote:
    Farkas Levente wrote:
    i simple state rhel/centos 5.3 can't be rebuild on itself without
    patching spec files and adding patches for a few files.
    And yet it is.
    did you try it?
    show me your build log the rebuild of ifd-egate-0.05-15.i386.rpm:-)
    Okay, that doesn't seem to have been rebuilt since 5.(0|1). But I assure
    you that this one has been built on a CentOS system.

    Ralph
    -------------- next part --------------
    A non-text attachment was scrubbed...
    Name: not available
    Type: application/pgp-signature
    Size: 189 bytes
    Desc: not available
    Url : http://lists.centos.org/pipermail/centos-devel/attachments/20090430/47ddb622/attachment.bin
  • R P Herrold at Apr 30, 2009 at 4:36 pm

    On Thu, 30 Apr 2009, Farkas Levente wrote:

    Ralph Angenendt wrote:
    Farkas Levente wrote:
    i simple state rhel/centos 5.3 can't be rebuild on itself without
    patching spec files and adding patches for a few files.
    And yet it is.
    did you try it?
    show me your build log the rebuild of ifd-egate-0.05-15.i386.rpm:-)
    Farkas, you win -- congratulations.

    ifd-egate-0.05-15.src.rpm does not currently build on a native
    5.3 fully updated; it formerly built under an earlier GCC
    variant in the centos-5 series. As such there has been a
    minor API change in what the gcc environment in question is
    accepting.

    If you want to know just which, step through all intermediate
    phases; the date on the binary file that IS in our archive
    tells you when to stop stepping forward through point releases
    and updates

    ifd-egate-0.05-17.src.rpm addreses this, builds on Centos 5.3
    current, and you may diff as you wish

    [herrold at centos-5 ifd-gate]$ rpm -qip \
    /home/herrold/rpmbuild/RPMS/x86_64/ifd-egate-0.05-17.x86_64.rpm
    Name : ifd-egate Relocations: (not relocatable)
    Version : 0.05 Vendor: orc
    Release : 17 Build Date: Thu 30 Apr 2009 12:34:00 PM EDT
    Install Date: (not installed) Build Host: centos-5.first.lan
    Group : System Environment/Base Source RPM: ifd-egate-0.05-17.src.rpm
    Size : 44909 License: BSD/LGPL
    Signature : (none)
    URL : http://secure.netroedge.com/~phil/egate/
    Summary : Axalto Egate SmartCard device driver for PCSC-lite
    Description :
    The Axalto Egate device driver enables PCSC-lite to
    communicate with Axalto Egate cards, which CoolKey is based
    off of.
    [herrold at centos-5 ifd-gate]$

    If you wish for me to research why, please contact me for a
    support contract, and I will research it, and file it upstream
    as 'regression' -- for more funds I'll even buy a licensed
    copy of RHEL, so I may open a CMS case; for even more funds,
    I'll buy enough mass that RedHat Rel Eng decides they want to
    fix it (note this may require buying the entire company's
    outstanding coommon stoc and will not be inexpensivek).

    Is there a point to this pedantry?

    -- Russ herrold
  • Les Mikesell at Apr 30, 2009 at 5:47 pm

    R P Herrold wrote:

    Is there a point to this pedantry?
    Is there a point to caring that you can rebuild _any_ source if you
    already have a binary that works?

    --
    Les Mikesell
    lesmikesell at gmail.com
  • Jeff Johnson at May 1, 2009 at 7:47 am

    On Apr 30, 2009, at 1:47 PM, Les Mikesell wrote:

    R P Herrold wrote:

    Is there a point to this pedantry?
    Is there a point to caring that you can rebuild _any_ source if you
    already have a binary that works?
    This is Hume's Problem of Induction:
    How do you know that the sun will rise tomorrow?
    Because the solar system "works"? Binary packages can break
    far more easily than the solar system.

    73 de Jeff
  • Mike A. Harris at Apr 30, 2009 at 9:57 am

    James Olin Oden wrote:

    Your spending a lot of time defending the status quo, but your not
    really answering an honest complaint. I've spent a lot of time
    servicing customers, and when they complain to you, they really don't
    care about explanations that don't try to help them with their
    immediate problem. Typically, at the point they start hearing long
    explanations of why things are the way they are without movement in
    the direction of helping them with their particular problems they just
    move on.

    I think the salient points are:

    * RedHat distros today don't self host.
    * Some customers desire that the distro could self host.

    Maybe the number of customers that desire this are too small to worry
    about, versus the cost of meeting their perceived needs. Maybe,
    solving the problem is really hard. None of that changes the above
    two facts; it only places them in a context.
    Depends on the precise definition one uses for "self host" I suppose.
    The only way to truly know that the entire set of current rpm packages
    compiles against itself, is for the entire OS to be rebuilt every time a
    single package changes, and if anything fails, then those packages have
    to be updated as well.

    I think it is very unreasonable to expect a large number of packages to
    be released as an update simply so that everything still compiles.
    There are already enough updates coming through the pipe as it is. To
    churn additional packages just for the moniker "self hosting" might be
    an amiable label to have, I don't really think it is practical.

    Having said that, what solution do you propose to this problem?
  • Farkas Levente at Apr 28, 2009 at 7:32 pm

    Mike A. Harris wrote:
    Les Mikesell wrote:
    James Antill wrote:
    just ot mention a few problem with 5.3:
    - openjava was added to the distro so all packages which requires
    java-devel now try to build with openjava in stead of gcc's java and
    most of them fail.
    - new updates like dbus-glib, ifd-gate, pccs etc have incompatible devel
    packages eg. headers, but not all of the packages requires these new
    packages was rebuild/fixed so those packages no longer build.
    - newer gcc, toolchain etc (which included in later updates) have
    stronger check and standard compliance but with these tools old and
    buggy code no longer compile.
    This is useless churn to rebuild all the packages to fix these kinds of
    build differences, why do you think RH's customers would want them to do
    that?
    I thought _THE_ selling point of open source has always been that in
    case of problems the vendor can't/won't fix, you have the option to make
    the change yourself. But if you can't rebuild their packages or even
    tell how the source relates to the shipped binary, that isn't true and
    shouldn't be represented as such.
    It sucks that the latest sources do not compile on the OS they were
    originally built for, but the fact is that packages get built at a
    certain point in time against what is in the tree at that time.

    Then packages continue to be updated to fix bugs, etc. and in some cases
    maybe even add a new feature here or there. All it takes is for one of
    these updates to change something in such a manner that packages that
    depend on this package will no longer compile. It could be as simple as
    a file being moved from one location to another, or some other innocent
    innocuous change.

    The only way to prevent that sort of thing is to have a mandatory policy
    that whenever _any_ package is rebuilt in the distribution, that the
    entire distribution must be rebuilt in a tree with the absolute latest
    packages present just to ensure that all packages still compile.
    no i don't think so. imho there are 2 stage:
    1. rebuild every package which depend on or require the new package if
    all build without error allow in the new packages otherwise don't allow
    until other compilation fails.
    2. same as 1. but also include all new rebuild in the updates which
    changes and here some smart diff required eg. binary must be the same
    some other files can differ eg. generated docs can have some 'small'
    differences.

    this requires a bit more build farm resource but produce a much better
    and stable distro.

    --
    Levente "Si vis pacem para bellum!"
  • James Olin Oden at Apr 28, 2009 at 7:35 pm

    On Tue, Apr 28, 2009 at 3:32 PM, Farkas Levente wrote:
    Mike A. Harris wrote:
    Les Mikesell wrote:
    James Antill wrote:
    just ot mention a few problem with 5.3:
    - openjava was added to the distro so all packages which requires
    java-devel now try to build with openjava in stead of gcc's java and
    most of them fail.
    - new updates like dbus-glib, ifd-gate, pccs etc have incompatible devel
    packages eg. headers, but not all of the packages requires these new
    packages was rebuild/fixed so those packages no longer build.
    - newer gcc, toolchain etc (which included in later updates) have
    stronger check and standard compliance but with these tools old and
    buggy code no longer compile.
    ?This is useless churn to rebuild all the packages to fix these kinds of
    build differences, why do you think RH's customers would want them to do
    that?
    I thought _THE_ selling point of open source has always been that in
    case of problems the vendor can't/won't fix, you have the option to make
    the change yourself. ?But if you can't rebuild their packages or even
    tell how the source relates to the shipped binary, that isn't true and
    shouldn't be represented as such.
    It sucks that the latest sources do not compile on the OS they were
    originally built for, but the fact is that packages get built at a
    certain point in time against what is in the tree at that time.

    Then packages continue to be updated to fix bugs, etc. and in some cases
    maybe even add a new feature here or there. ?All it takes is for one of
    these updates to change something in such a manner that packages that
    depend on this package will no longer compile. ?It could be as simple as
    a file being moved from one location to another, or some other innocent
    innocuous change.

    The only way to prevent that sort of thing is to have a mandatory policy
    that whenever _any_ package is rebuilt in the distribution, that the
    entire distribution must be rebuilt in a tree with the absolute latest
    packages present just to ensure that all packages still compile.
    no i don't think so. imho there are 2 stage:
    1. rebuild every package which depend on or require the new package if
    all build without error allow in the new packages otherwise don't allow
    until other compilation fails.
    2. same as 1. but also include all new rebuild in the updates which
    changes and here some smart diff required eg. binary must be the same
    some other files can differ eg. generated docs can have some 'small'
    differences.

    this requires a bit more build farm resource but produce a much better
    and stable distro.
    Right and that's a business decision not a technical decision....james
  • Karanbir Singh at Apr 28, 2009 at 11:13 pm
    Les,

    Les Mikesell wrote:
    I thought _THE_ selling point of open source has always been that in
    case of problems the vendor can't/won't fix, you have the option to make
    the change yourself. But if you can't rebuild their packages or even
    tell how the source relates to the shipped binary, that isn't true and
    shouldn't be represented as such.
    You are working on the assumption that you cant build the packages....
    which I can assure you is not try. Look, there is this thing called
    CentOS which should prove that its possible to rebuild and build exactly
    whats happening upstream too :)

    --
    Karanbir Singh : http://www.karan.org/ : 2522219 at icq
  • Dag Wieers at Apr 29, 2009 at 1:03 am

    On Wed, 29 Apr 2009, Karanbir Singh wrote:

    Les Mikesell wrote:
    I thought _THE_ selling point of open source has always been that in
    case of problems the vendor can't/won't fix, you have the option to make
    the change yourself. But if you can't rebuild their packages or even
    tell how the source relates to the shipped binary, that isn't true and
    shouldn't be represented as such.
    You are working on the assumption that you cant build the packages....
    which I can assure you is not try. Look, there is this thing called
    CentOS which should prove that its possible to rebuild and build exactly
    whats happening upstream too :)
    What is even more, I would like to welcome any Open Source project
    rebuilding RHEL. Not only do I think this would provide healthy
    competition/collaboration between projects (again), different projects may
    tackle problems differently, adding to the ecosystem. Questions about the
    process would bring clarity and may help with better documentation of the
    RHEL build process, validation and QA.

    A new rebuild project also does not impose any limits that new users
    experience when contributing to the core of the CentOS project.
    (limitations related to trust, signing, empowerment, mindset, ...)

    And I hope a project rethinking the rebuild process without the
    baggage and continuous effort/strain to produce update release
    may be more practical too.

    The success of CentOS caused some other projects to fade away, which I
    think is unfortunate but understandable.

    --
    -- dag wieers, dag at centos.org, http://dag.wieers.com/ --
    [Any errors in spelling, tact or fact are transmission errors]
  • James Olin Oden at Apr 28, 2009 at 4:53 pm

    On Mon, Apr 27, 2009 at 10:56 AM, James Antill wrote:
    On Fri, 2009-04-24 at 23:55 +0200, Farkas Levente wrote:

    just ot mention a few problem with 5.3:
    - openjava was added to the distro so all packages which requires
    java-devel now try to build with openjava in stead of gcc's java and
    most of them fail.
    - new updates like dbus-glib, ifd-gate, pccs etc have incompatible devel
    packages eg. headers, but not all of the packages requires these new
    packages was rebuild/fixed so those packages no longer build.
    - newer gcc, toolchain etc (which included in later updates) have
    stronger check and standard compliance but with these tools old and
    buggy code no longer compile.
    ?This is useless churn to rebuild all the packages to fix these kinds of
    build differences, why do you think RH's customers would want them to do
    that?
    One man's uselessness is another man's requirement. Also, when making
    blanket statements about customers, realize that he too is customer,
    and he clearly has a different needs. He is not alone, either.
    RedHat being able to self host itself is very important to the class
    of customer that is customizing RedHat distributions. If it can't
    self host itself, the process of building all of RedHat being
    documented would be an acceptable alternative. As an aside, if they
    didn't use these new fangled compilers, which ones did they use?
    That is followed by if they didn't use the new compilers to build
    RedHat, then why should we as customers trust these compilers for our
    products?

    Also, consider that many of us use open source so we can quickly and
    effectively support our customers. That benefit is derived solely
    from the access to the source code, and our ability to patch and
    rebuild programs from these sources. If the ability to rebuild from
    sources is not there or undocumented, then naturally, we start looking
    at other alternatives.

    Cheers...james

Related Discussions

People

Translate

site design / logo © 2022 Grokbase