The way it is now (9.2.2):

1. set up a new system
2. forget to install libperl-dev
3. ./configure --with-perl
4. get no failures or warnings
5. make
6. make errors out on plperl.so

The way it should work:

- configure should error out, as it does with python.

--
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com

Search Discussions

  • Tom Lane at Jan 9, 2013 at 2:10 am

    Josh Berkus writes:
    The way it is now (9.2.2):
    1. set up a new system
    2. forget to install libperl-dev
    3. ./configure --with-perl
    4. get no failures or warnings
    5. make
    6. make errors out on plperl.so
    The way it should work:
    - configure should error out, as it does with python.
    Hm. I don't see any configure-time test for availability of perl.h,
    which is probably what we ought to add here.

        regards, tom lane
  • Tom Lane at Jan 9, 2013 at 3:38 am

    Josh Berkus writes:
    The way it is now (9.2.2):
    1. set up a new system
    2. forget to install libperl-dev
    3. ./configure --with-perl
    4. get no failures or warnings
    5. make
    6. make errors out on plperl.so
    The way it should work:
    - configure should error out, as it does with python.
    I poked at this a bit more, and AFAICS your real beef is with Debian's
    whacked-out packaging decisions for Perl.

    Item: it's impossible to reproduce this failure on Red Hat-based
    distros. perl-devel is required by perl-ExtUtils-MakeMaker, without
    which we'll definitely fail at configure time. I doubt it's possible
    in a hand-rolled install from source, either, because all that stuff
    is part of the core perl tarball.

    Item: there is *not* any test for libpython.so, as such, in our
    configure script. There is a test for Python.h, which is sufficient on
    Red Hat systems as well as on Debian, because python-devel
    (resp. python-dev) carries both the header files and the .so links.

    Item: there is not a test for perl.h, as such, in configure. There
    probably should be, just because we have comparable tests for tcl.h
    and Python.h. However, adding one won't fix your problem on
    Debian-based distros, because for some wacko reason they put the
    headers and the shlib .so symlink in different packages, cf
    http://packages.debian.org/squeeze/amd64/perl/filelist
    http://packages.debian.org/squeeze/amd64/libperl-dev/filelist
    I am unfamiliar with a good reason for doing that. (I can certainly
    see segregating the .a static library, or even not shipping it at
    all, but what's it save to leave out the .so symlink?)

    We could try adding an AC_TRY_LINK test using perl_embed_ldflags,
    but given the weird stuff happening to redefine that value on Windows
    in plperl/GNUmakefile I think there's a serious risk of breaking Cygwin
    builds. Since I lack access to either Cygwin or a platform on which
    there's a problem today, I'm not going to be the one to mess with it.

        regards, tom lane
  • Peter Eisentraut at Jan 9, 2013 at 4:33 am

    On Tue, 2013-01-08 at 22:37 -0500, Tom Lane wrote:
    We could try adding an AC_TRY_LINK test using perl_embed_ldflags,
    but given the weird stuff happening to redefine that value on Windows
    in plperl/GNUmakefile I think there's a serious risk of breaking
    Cygwin builds. Since I lack access to either Cygwin or a platform on
    which there's a problem today, I'm not going to be the one to mess
    with it.
    For these and similar reasons, I think it's best not to mess with this
    too much. It should fail where it actually fails. We shouldn't be
    duplicating that logic in a remote place to predict that something will
    fail later. In some cases, this is rather safe and easy and useful as a
    service to the user, but once it gets slightly complicated, it can only
    go wrong.
  • Josh Berkus at Jan 9, 2013 at 6:02 am
    Tom,
    I poked at this a bit more, and AFAICS your real beef is with Debian's
    whacked-out packaging decisions for Perl.
    Ah, I do so much Ubuntu these days I didn't recognize the pattern.

    I'll try to figure out a good place to document a warning about it.

    --
    Josh Berkus
    PostgreSQL Experts Inc.
    http://pgexperts.com
  • Andrew Dunstan at Jan 9, 2013 at 8:18 am

    On 01/08/2013 10:37 PM, Tom Lane wrote:

    We could try adding an AC_TRY_LINK test using perl_embed_ldflags,
    but given the weird stuff happening to redefine that value on Windows
    in plperl/GNUmakefile I think there's a serious risk of breaking Cygwin
    builds. Since I lack access to either Cygwin or a platform on which
    there's a problem today, I'm not going to be the one to mess with it.

    ITYM Mingw - the Makefile doesn't do anything for Cygwin.

    If you want to build a configure test, you could make it conditional on
    the PORTNAME not being win32, since we don't seem to have a problem
    there anyway.

    cheers

    andrew
  • Tom Lane at Jan 9, 2013 at 3:18 pm

    Andrew Dunstan writes:
    On 01/08/2013 10:37 PM, Tom Lane wrote:
    We could try adding an AC_TRY_LINK test using perl_embed_ldflags,
    but given the weird stuff happening to redefine that value on Windows
    in plperl/GNUmakefile I think there's a serious risk of breaking Cygwin
    builds. Since I lack access to either Cygwin or a platform on which
    there's a problem today, I'm not going to be the one to mess with it.
    ITYM Mingw - the Makefile doesn't do anything for Cygwin.
    OK, sorry.
    If you want to build a configure test, you could make it conditional on
    the PORTNAME not being win32, since we don't seem to have a problem
    there anyway.
    Actually, if we were to try to clean this up, I'd suggest moving that
    logic into the configure script --- it's not apparent to me why it's
    a good idea to be changing configure-determined values in the Makefile.
    But in any case this would have to be done by somebody who's in a
    position to test on affected platforms.

        regards, tom lane
  • Andrew Dunstan at Jan 9, 2013 at 9:06 pm

    On 01/09/2013 10:16 AM, Tom Lane wrote:
    Andrew Dunstan <andrew@dunslane.net> writes:
    On 01/08/2013 10:37 PM, Tom Lane wrote:
    We could try adding an AC_TRY_LINK test using perl_embed_ldflags,
    but given the weird stuff happening to redefine that value on Windows
    in plperl/GNUmakefile I think there's a serious risk of breaking Cygwin
    builds. Since I lack access to either Cygwin or a platform on which
    there's a problem today, I'm not going to be the one to mess with it.
    ITYM Mingw - the Makefile doesn't do anything for Cygwin.
    OK, sorry.
    If you want to build a configure test, you could make it conditional on
    the PORTNAME not being win32, since we don't seem to have a problem
    there anyway.
    Actually, if we were to try to clean this up, I'd suggest moving that
    logic into the configure script --- it's not apparent to me why it's
    a good idea to be changing configure-determined values in the Makefile.
    But in any case this would have to be done by somebody who's in a
    position to test on affected platforms.

    Here's a patch which does that and produces configure traces like this
    on Mingw:

         checking for Perl archlibexp... C:/Perl/lib
         checking for Perl privlibexp... C:/Perl/lib
         checking for Perl useshrplib... true
         checking for flags to link embedded Perl... -LC:/Perl/lib/CORE -lperl512

    which seems to be what we want.

    Given that, you should be able to write a reasonably portable configure
    test for library presence.

    Barring objection I'll apply this shortly.

    cheers

    andrew
  • Tom Lane at Jan 9, 2013 at 9:12 pm

    Andrew Dunstan writes:
    On 01/09/2013 10:16 AM, Tom Lane wrote:
    Actually, if we were to try to clean this up, I'd suggest moving that
    logic into the configure script --- it's not apparent to me why it's
    a good idea to be changing configure-determined values in the Makefile.
    But in any case this would have to be done by somebody who's in a
    position to test on affected platforms.
    Here's a patch which does that and produces configure traces like this
    on Mingw:
    checking for Perl archlibexp... C:/Perl/lib
    checking for Perl privlibexp... C:/Perl/lib
    checking for Perl useshrplib... true
    checking for flags to link embedded Perl... -LC:/Perl/lib/CORE -lperl512
    which seems to be what we want.
    Given that, you should be able to write a reasonably portable configure
    test for library presence.
    Looks good. If you're happy with that then I can undertake to add a
    libperl.so probe based on AC_TRY_LINK with the unmodified value of
    $perl_embed_ldflags.

        regards, tom lane
  • Andrew Dunstan at Jan 9, 2013 at 9:49 pm

    On 01/09/2013 04:12 PM, Tom Lane wrote:
    Andrew Dunstan <andrew@dunslane.net> writes:
    On 01/09/2013 10:16 AM, Tom Lane wrote:
    Actually, if we were to try to clean this up, I'd suggest moving that
    logic into the configure script --- it's not apparent to me why it's
    a good idea to be changing configure-determined values in the Makefile.
    But in any case this would have to be done by somebody who's in a
    position to test on affected platforms.
    Here's a patch which does that and produces configure traces like this
    on Mingw:
    checking for Perl archlibexp... C:/Perl/lib
    checking for Perl privlibexp... C:/Perl/lib
    checking for Perl useshrplib... true
    checking for flags to link embedded Perl... -LC:/Perl/lib/CORE -lperl512
    which seems to be what we want.
    Given that, you should be able to write a reasonably portable configure
    test for library presence.
    Looks good. If you're happy with that then I can undertake to add a
    libperl.so probe based on AC_TRY_LINK with the unmodified value of
    $perl_embed_ldflags.

    Go for it.

    cheers

    andrew
  • Tom Lane at Jan 10, 2013 at 12:57 am

    Andrew Dunstan writes:
    On 01/09/2013 04:12 PM, Tom Lane wrote:
    Looks good. If you're happy with that then I can undertake to add a
    libperl.so probe based on AC_TRY_LINK with the unmodified value of
    $perl_embed_ldflags.
    Go for it.
    Done, we'll soon see how much the buildfarm likes it. (I did test on
    Fedora, OS X, and a from-source build on HPUX, though.)

    While attempting to test this, I found out that my previous statement
    that this failure is impossible on Red Hat platforms actually needs no
    qualification. I had supposed that it would be possible to test the
    failure path with sufficiently aggressive damage to the local perl
    installation, but it's not. Observe:

    $ which perl
    /usr/bin/perl
    $ ldd /usr/bin/perl
             linux-vdso.so.1 => (0x00007fffe03ff000)
             libperl.so => /usr/lib64/perl5/CORE/libperl.so (0x0000003c0d400000)
             libresolv.so.2 => /lib64/libresolv.so.2 (0x0000003831600000)
             libnsl.so.1 => /lib64/libnsl.so.1 (0x0000003843600000)
             libdl.so.2 => /lib64/libdl.so.2 (0x000000382fe00000)
             libm.so.6 => /lib64/libm.so.6 (0x000000382fa00000)
             libcrypt.so.1 => /lib64/libcrypt.so.1 (0x0000003c0d800000)
             libutil.so.1 => /lib64/libutil.so.1 (0x0000003842a00000)
             libpthread.so.0 => /lib64/libpthread.so.0 (0x000000382f600000)
             libc.so.6 => /lib64/libc.so.6 (0x000000382f200000)
             /lib64/ld-linux-x86-64.so.2 (0x000000382ee00000)
             libfreebl3.so => /lib64/libfreebl3.so (0x0000003c0e000000)
    $ ls -l /usr/lib64/perl5/CORE/libperl*
    -rwxr-xr-x. 1 root root 1552816 Nov 26 10:25 /usr/lib64/perl5/CORE/libperl.so

    That is, the standard perl executable depends on libperl.so --- not
    libperl.so.M.N, which isn't even there. Take the .so away, and you
    don't get past configure's initial perl-related checks, because
    /usr/bin/perl is broken. This isn't a Red-Hat-ism, either, because my
    direct-from-upstream-source build on my old HPUX box is the same way.

    I conclude that Debian has positively striven to be randomly different
    from the rest of the Perl world, and so my sympathy for their packaging
    design is even lower than it was this morning.

        regards, tom lane
  • Tom Lane at Jan 10, 2013 at 3:48 am

    I wrote:
    Done, we'll soon see how much the buildfarm likes it.
    Well, okapi (Gentoo) doesn't like it:

    configure:29191: checking for libperl
    configure:29217: icc -o conftest -O3 -xSSSE3 -parallel -ip -mp1 -fno-strict-aliasing -g -I/usr/lib64/perl5/5.12.4/x86_64-linux/CORE -D_GNU_SOURCE -I/usr/include/libxml2 -I/usr/include/et conftest.c -Wl,-O1 -Wl,--as-needed -L/usr/lib64/perl5/5.12.4/x86_64-linux/CORE -lperl -lnsl -ldl -lm -lcrypt -lutil -lc -lxslt -lxml2 -lssl -lcrypto -lkrb5 -lz -lreadline -lcrypt -lm >&5
    ld: conftest: hidden symbol `pthread_atfork' in /usr/lib64/libpthread_nonshared.a(pthread_atfork.oS) is referenced by DSO
    ld: final link failed: Bad value

    The previous successful build on that box shows that plperl was linked
    like this without any error:

    icc -O3 -xSSSE3 -parallel -ip -mp1 -fno-strict-aliasing -g -fpic -shared -o plperl.so plperl.o SPI.o Util.o -L../../../src/port -Wl,-rpath,'/usr/lib64/perl5/5.12.4/x86_64-linux/CORE',--enable-new-dtags -Wl,-O1 -Wl,--as-needed -L/usr/lib64/perl5/5.12.4/x86_64-linux/CORE -lperl -lnsl -ldl -lm -lcrypt -lutil -lc

    The best guess I can come up with is that this box is only able to link
    libperl.so into a shared library, ie the "-fpic -shared" part of the
    recipe is critical. If so, this idea is a failure and we're gonna have
    to remove the link test, because there is no way I'm going to try to put
    enough smarts into the configure script to do a shared-library link
    correctly.

    However, I don't know Gentoo at all, and it may be there's some other
    explanation that we could cope with more readily. Thoughts?

        regards, tom lane
  • Andrew Dunstan at Jan 10, 2013 at 2:57 pm

    On 01/09/2013 10:48 PM, Tom Lane wrote:
    I wrote:
    Done, we'll soon see how much the buildfarm likes it.
    Well, okapi (Gentoo) doesn't like it:

    configure:29191: checking for libperl
    configure:29217: icc -o conftest -O3 -xSSSE3 -parallel -ip -mp1 -fno-strict-aliasing -g -I/usr/lib64/perl5/5.12.4/x86_64-linux/CORE -D_GNU_SOURCE -I/usr/include/libxml2 -I/usr/include/et conftest.c -Wl,-O1 -Wl,--as-needed -L/usr/lib64/perl5/5.12.4/x86_64-linux/CORE -lperl -lnsl -ldl -lm -lcrypt -lutil -lc -lxslt -lxml2 -lssl -lcrypto -lkrb5 -lz -lreadline -lcrypt -lm >&5
    ld: conftest: hidden symbol `pthread_atfork' in /usr/lib64/libpthread_nonshared.a(pthread_atfork.oS) is referenced by DSO
    ld: final link failed: Bad value

    The previous successful build on that box shows that plperl was linked
    like this without any error:

    icc -O3 -xSSSE3 -parallel -ip -mp1 -fno-strict-aliasing -g -fpic -shared -o plperl.so plperl.o SPI.o Util.o -L../../../src/port -Wl,-rpath,'/usr/lib64/perl5/5.12.4/x86_64-linux/CORE',--enable-new-dtags -Wl,-O1 -Wl,--as-needed -L/usr/lib64/perl5/5.12.4/x86_64-linux/CORE -lperl -lnsl -ldl -lm -lcrypt -lutil -lc

    The best guess I can come up with is that this box is only able to link
    libperl.so into a shared library, ie the "-fpic -shared" part of the
    recipe is critical. If so, this idea is a failure and we're gonna have
    to remove the link test, because there is no way I'm going to try to put
    enough smarts into the configure script to do a shared-library link
    correctly.
    This seems fairly unlikely. You're supposed to be able to use it in a
    standalone executable - see the perlembed manpage. But I guess with
    Gentoo anything is possible.
    However, I don't know Gentoo at all, and it may be there's some other
    explanation that we could cope with more readily. Thoughts?

    Maybe we need a test that mirrors our use of PERL_SYS_INIT3?, something
    like:

         #include <EXTERN.h> /* from the Perl distribution */
         #include <perl.h> /* from the Perl distribution */

         static PerlInterpreter *my_perl; /*** The Perl interpreter ***/

         int main(int argc, char **argv, char **env)
         {
         #if defined(PERL_SYS_INIT3) && !defined(MYMALLOC)
              PERL_SYS_INIT3(&argc,&argv,&env);
         #endif
              my_perl = perl_alloc();
         }


    Or maybe for a standalone executable we need to use the ccopts as
    suggested by the perlembed manpage:

          cc -o interp interp.c `perl -MExtUtils::Embed -e ccopts -e ldopts`

    cheers

    andrew
  • Tom Lane at Jan 10, 2013 at 4:32 pm

    Andrew Dunstan writes:
    On 01/09/2013 10:48 PM, Tom Lane wrote:
    Well, okapi (Gentoo) doesn't like it:
    ...
    The best guess I can come up with is that this box is only able to link
    libperl.so into a shared library, ie the "-fpic -shared" part of the
    recipe is critical.
    This seems fairly unlikely. You're supposed to be able to use it in a
    standalone executable - see the perlembed manpage. But I guess with
    Gentoo anything is possible.
    It seemed pretty silly to me too. After posting it occurred to me that
    maybe the problem wasn't from libperl but one of the other LIBS that
    configure dredges up, but my fix along that line hasn't helped. So
    I don't know what to think, other than that Gentoo is fragile as can be.
    Maybe we need a test that mirrors our use of PERL_SYS_INIT3?
    I don't know what PERL_SYS_INIT3 expands to, but it's hard to see how it
    would expand to something that would fix this. The problem looks to be
    that something is pulling in libpthread and getting the wrong version.
    Or maybe for a standalone executable we need to use the ccopts as
    suggested by the perlembed manpage:
    cc -o interp interp.c `perl -MExtUtils::Embed -e ccopts -e ldopts`
    Hm. It would be interesting to see what ccopts expands to, but if you
    compare the command configure issued with the previous successful link
    of plperl, there's nothing missing that ccopts would be likely to
    supply.

        regards, tom lane
  • Andrew Dunstan at Jan 10, 2013 at 4:38 pm

    On 01/10/2013 11:32 AM, Tom Lane wrote:
    Andrew Dunstan <andrew@dunslane.net> writes:
    On 01/09/2013 10:48 PM, Tom Lane wrote:
    Well, okapi (Gentoo) doesn't like it:
    ...
    The best guess I can come up with is that this box is only able to link
    libperl.so into a shared library, ie the "-fpic -shared" part of the
    recipe is critical.
    This seems fairly unlikely. You're supposed to be able to use it in a
    standalone executable - see the perlembed manpage. But I guess with
    Gentoo anything is possible.
    It seemed pretty silly to me too. After posting it occurred to me that
    maybe the problem wasn't from libperl but one of the other LIBS that
    configure dredges up, but my fix along that line hasn't helped. So
    I don't know what to think, other than that Gentoo is fragile as can be.
    Maybe we need a test that mirrors our use of PERL_SYS_INIT3?
    I don't know what PERL_SYS_INIT3 expands to, but it's hard to see how it
    would expand to something that would fix this. The problem looks to be
    that something is pulling in libpthread and getting the wrong version.
    Or maybe for a standalone executable we need to use the ccopts as
    suggested by the perlembed manpage:
    cc -o interp interp.c `perl -MExtUtils::Embed -e ccopts -e ldopts`
    Hm. It would be interesting to see what ccopts expands to, but if you
    compare the command configure issued with the previous successful link
    of plperl, there's nothing missing that ccopts would be likely to
    supply.
    Without access to the machine it's pretty hard to know, so I was just
    speculating fairly wildly. If Jeremy can find out what the problem is
    that would be good, or if he can give us access to the machine it
    shouldn't be terribly hard to get to the bottom.

    cheers

    andrew
  • Pgbuildfarm at Jan 11, 2013 at 5:30 am
    On Thu, 10 Jan 2013, Andrew Dunstan wrote:
    On 01/10/2013 11:32 AM, Tom Lane wrote:
    Andrew Dunstan <andrew@dunslane.net> writes:
    cc -o interp interp.c `perl -MExtUtils::Embed -e ccopts -e ldopts`
    Hm. It would be interesting to see what ccopts expands to, but if you
    compare the command configure issued with the previous successful link
    of plperl, there's nothing missing that ccopts would be likely to
    supply.
    $ perl -MExtUtils::Embed -e ccopts -e ldopts
    -Wl,-E -Wl,-O1 -Wl,--as-needed
    -L/usr/lib64/perl5/5.12.4/x86_64-linux/CORE -lperl -lnsl -ldl -lm -lcrypt
    -lutil -lc
      -fno-strict-aliasing -pipe -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64
    -I/usr/lib64/perl5/5.12.4/x86_64-linux/CORE
    Without access to the machine it's pretty hard to know, so I was just
    speculating fairly wildly. If Jeremy can find out what the problem is that
    would be good, or if he can give us access to the machine it shouldn't be
    terribly hard to get to the bottom.
    I'll see what I can do. For now, I can tell you that it works with GCC
    and fails with ICC.
  • Tom Lane at Jan 11, 2013 at 5:48 am

    pgbuildfarm@jdrake.com writes:
    On Thu, 10 Jan 2013, Andrew Dunstan wrote:
    Without access to the machine it's pretty hard to know, so I was just
    speculating fairly wildly. If Jeremy can find out what the problem is that
    would be good, or if he can give us access to the machine it shouldn't be
    terribly hard to get to the bottom.
    I'll see what I can do. For now, I can tell you that it works with GCC
    and fails with ICC.
    Well, that's darn interesting in itself, because the error message looks
    like it should be purely a linker issue. (And I note that your other
    buildfarm animal mongoose uses icc but is working anyway, so that's
    definitely not the whole story ...)

    Please note Aaron Swenson's offer of help too -- he's probably a lot
    better qualified than anybody else here to figure out what is going on
    with this.

        regards, tom lane
  • Jeremy Drake at Jan 11, 2013 at 6:01 am

    On Fri, 11 Jan 2013, Tom Lane wrote:

    pgbuildfarm@jdrake.com writes:
    Well, that's darn interesting in itself, because the error message looks
    like it should be purely a linker issue. (And I note that your other
    buildfarm animal mongoose uses icc but is working anyway, so that's
    definitely not the whole story ...)
    mongoose is 32-bit, and a really old version of icc. okapi is 64-bit, and
    a merely moderately old icc. I should set up a dedicated buildfarm
    VM with the latest version...
    Please note Aaron Swenson's offer of help too -- he's probably a lot
    better qualified than anybody else here to figure out what is going on
    with this.
    I'm sorry, I didn't see this. It must not have been CC'd to me, I don't
    subscribe to -hackers anymore, I just couldn't keep up with the traffic
    after I got a new job that wasn't postgres-related.
  • Tom Lane at Jan 14, 2013 at 5:09 pm

    pgbuildfarm@jdrake.com writes:
    On Thu, 10 Jan 2013, Andrew Dunstan wrote:
    Without access to the machine it's pretty hard to know, so I was just
    speculating fairly wildly. If Jeremy can find out what the problem is that
    would be good, or if he can give us access to the machine it shouldn't be
    terribly hard to get to the bottom.
    I'll see what I can do. For now, I can tell you that it works with GCC
    and fails with ICC.
    BTW, just to close out this thread on the public list --- Andrew found
    out that the problem goes away if one removes "-parallel" from the list
    of compiler options. That makes it a pretty blatant compiler bug, and
    one that we've not seen elsewhere, so it's probably specific to the
    (old) icc version that Jeremy is running on this buildfarm animal.

    Given that no other buildfarm members are showing a problem, I don't
    intend to revert the configure change. Jeremy should update his icc
    or remove the -parallel switch on that animal.

        regards, tom lane
  • Peter Eisentraut at Jan 10, 2013 at 9:21 pm

    On 1/9/13 7:56 PM, Tom Lane wrote:
    That is, the standard perl executable depends on libperl.so --- not
    libperl.so.M.N, which isn't even there. Take the .so away, and you
    don't get past configure's initial perl-related checks, because
    /usr/bin/perl is broken. This isn't a Red-Hat-ism, either, because my
    direct-from-upstream-source build on my old HPUX box is the same way.
    How does this survive a distribution upgrade with a new libperl soname?
  • Tom Lane at Jan 10, 2013 at 9:48 pm

    Peter Eisentraut writes:
    On 1/9/13 7:56 PM, Tom Lane wrote:
    That is, the standard perl executable depends on libperl.so --- not
    libperl.so.M.N, which isn't even there. Take the .so away, and you
    don't get past configure's initial perl-related checks, because
    /usr/bin/perl is broken. This isn't a Red-Hat-ism, either, because my
    direct-from-upstream-source build on my old HPUX box is the same way.
    How does this survive a distribution upgrade with a new libperl soname?
    Well, I'm not the package maintainer for perl, so this is not an
    authoritative answer ... but I don't believe that there's any
    expectation that you could replace the installation with a different
    major perl version and still have C-level dependencies work. Adding a
    soname number wouldn't exactly be enough to fix that type of problem,
    anyhow, considering the number of ABI-incompatible ways you can build
    libperl.

    There is some version dependency checking, all right, but it's done off
    of special RPM provides/requires symbols not the library soname. For
    instance I see this on a Fedora build of plperl:

    $ rpm -qp postgresql-plperl-9.2.2-3.fc16.x86_64.rpm --requires
    libc.so.6()(64bit)
    libc.so.6(GLIBC_2.2.5)(64bit)
    libc.so.6(GLIBC_2.3)(64bit)
    libc.so.6(GLIBC_2.3.4)(64bit)
    libc.so.6(GLIBC_2.4)(64bit)
    libperl.so()(64bit)
    libpthread.so.0()(64bit)
    libpthread.so.0(GLIBC_2.2.5)(64bit)
    perl(:MODULE_COMPAT_5.14.3) <-----------------
    postgresql-server(x86-64) = 9.2.2-3.fc16
    rpmlib(CompressedFileNames) <= 3.0.4-1
    rpmlib(PayloadFilesHavePrefix) <= 4.0-1
    rtld(GNU_HASH)

    I suspect they chose that method so it would do something useful for
    Perl extension modules as well as programs that require libperl.

        regards, tom lane
  • Peter Eisentraut at Jan 10, 2013 at 10:05 pm

    On 1/10/13 4:48 PM, Tom Lane wrote:
    Well, I'm not the package maintainer for perl, so this is not an
    authoritative answer ... but I don't believe that there's any
    expectation that you could replace the installation with a different
    major perl version and still have C-level dependencies work.
    Well, Debian does support that. It is necessary to be able to upgrade
    to the next distribution release and have old packages linked against an
    older libperl keep working. Apparently, Fedora doesn't support that.
  • Tom Lane at Jan 10, 2013 at 10:30 pm

    Peter Eisentraut writes:
    On 1/10/13 4:48 PM, Tom Lane wrote:
    Well, I'm not the package maintainer for perl, so this is not an
    authoritative answer ... but I don't believe that there's any
    expectation that you could replace the installation with a different
    major perl version and still have C-level dependencies work.
    Well, Debian does support that. It is necessary to be able to upgrade
    to the next distribution release and have old packages linked against an
    older libperl keep working. Apparently, Fedora doesn't support that.
    If they wanted to install two incompatible versions at once, they'd just
    stick them in different directory trees (ie, not both /usr/lib64/perl5).
    Given the amount of stuff in a Perl distribution besides libperl.so
    itself, I should think some such thing would be necessary regardless.

    (Or, if you prefer, the thing that is binding /usr/bin/perl to its
    correct library is an RPATH setting, not a soname version number.)

        regards, tom lane
  • Josh Berkus at Jan 11, 2013 at 12:53 am
    All,

    OK, now I'm sorry I brought this up.


    --
    Josh Berkus
    PostgreSQL Experts Inc.
    http://pgexperts.com
  • Tom Lane at Jan 11, 2013 at 1:13 am

    Josh Berkus writes:
    OK, now I'm sorry I brought this up.
    Sorry about going off on a theological tangent. The important question
    at this point is, can anybody fix the test so it works on Gentoo?
    If not we'll have to pull it out, whether or not it would be a good
    thing to have on Debian.

        regards, tom lane
  • Aaron W. Swenson at Jan 11, 2013 at 3:09 am

    On Thu, Jan 10, 2013 at 08:12:56PM -0500, Tom Lane wrote:
    Josh Berkus <josh@agliodbs.com> writes:
    OK, now I'm sorry I brought this up.
    Sorry about going off on a theological tangent. The important question
    at this point is, can anybody fix the test so it works on Gentoo?
    If not we'll have to pull it out, whether or not it would be a good
    thing to have on Debian.

    regards, tom lane
    I can give it a go.

    At any rate, I'll have to fix the problem before I
    can release the package to the Portage tree, even if that means I remove
    the test.

    --
    Mr. Aaron W. Swenson
    Gentoo Linux Developer
    Email : titanofold@gentoo.org
    GnuPG FP : 2C00 7719 4F85 FB07 A49C 0E31 5713 AA03 D1BB FDA0
    GnuPG ID : D1BBFDA0
  • Tom Lane at Jan 11, 2013 at 3:25 am

    "Aaron W. Swenson" <titanofold@gentoo.org> writes:
    On Thu, Jan 10, 2013 at 08:12:56PM -0500, Tom Lane wrote:
    Sorry about going off on a theological tangent. The important question
    at this point is, can anybody fix the test so it works on Gentoo?
    If not we'll have to pull it out, whether or not it would be a good
    thing to have on Debian.
    I can give it a go.
    Great, thanks!

    One thing we can see now that we have a full buildfarm cycle with that
    patch is that two out of the three Gentoo machines in the farm are
    perfectly happy with it --- only okapi doesn't like it. So that raises
    a new set of questions. It seems somewhat likely now that this is a
    local misconfiguration problem on okapi, but unless we can diagnose it
    I still feel that we'll have to drop the configure probe.

    Just a moment ago I pushed another change to make configure place the
    -I$perl_archlibexp/CORE switch at the end of CPPFLAGS not the beginning,
    because that's how plperl's makefile does it. I have not got much hope
    that the problem is an include search order issue, but that's the last
    thing I can think to try. I don't see any other plausibly-significant
    differences between what configure is doing and what the makefile is
    doing.

        regards, tom lane
  • Aaron W. Swenson at Jan 11, 2013 at 3:59 am

    On Thu, Jan 10, 2013 at 10:25:25PM -0500, Tom Lane wrote:
    "Aaron W. Swenson" <titanofold@gentoo.org> writes:
    On Thu, Jan 10, 2013 at 08:12:56PM -0500, Tom Lane wrote:
    Sorry about going off on a theological tangent. The important question
    at this point is, can anybody fix the test so it works on Gentoo?
    If not we'll have to pull it out, whether or not it would be a good
    thing to have on Debian.
    I can give it a go.
    Great, thanks!

    One thing we can see now that we have a full buildfarm cycle with that
    patch is that two out of the three Gentoo machines in the farm are
    perfectly happy with it --- only okapi doesn't like it. So that raises
    a new set of questions. It seems somewhat likely now that this is a
    local misconfiguration problem on okapi, but unless we can diagnose it
    I still feel that we'll have to drop the configure probe.

    Just a moment ago I pushed another change to make configure place the
    -I$perl_archlibexp/CORE switch at the end of CPPFLAGS not the beginning,
    because that's how plperl's makefile does it. I have not got much hope
    that the problem is an include search order issue, but that's the last
    thing I can think to try. I don't see any other plausibly-significant
    differences between what configure is doing and what the makefile is
    doing.

    regards, tom lane
    Just checked out that change and it works for me. It finds libperl.

    --
    Mr. Aaron W. Swenson
    Gentoo Linux Developer
    Email : titanofold@gentoo.org
    GnuPG FP : 2C00 7719 4F85 FB07 A49C 0E31 5713 AA03 D1BB FDA0
    GnuPG ID : D1BBFDA0
  • Peter Eisentraut at Jan 10, 2013 at 9:19 pm

    On 1/9/13 4:06 PM, Andrew Dunstan wrote:
    Here's a patch which does that and produces configure traces like this
    on Mingw:

    checking for Perl archlibexp... C:/Perl/lib
    checking for Perl privlibexp... C:/Perl/lib
    checking for Perl useshrplib... true
    checking for flags to link embedded Perl... -LC:/Perl/lib/CORE -lperl512

    which seems to be what we want.
    It would also be good to fix the same issue in plpython.
  • Christoph Berg at Jan 9, 2013 at 10:21 am
    Re: Tom Lane 2013-01-09 <9802.1357702675@sss.pgh.pa.us>
    Item: there is not a test for perl.h, as such, in configure. There
    probably should be, just because we have comparable tests for tcl.h
    and Python.h. However, adding one won't fix your problem on
    Debian-based distros, because for some wacko reason they put the
    headers and the shlib .so symlink in different packages, cf
    http://packages.debian.org/squeeze/amd64/perl/filelist
    http://packages.debian.org/squeeze/amd64/libperl-dev/filelist
    I am unfamiliar with a good reason for doing that. (I can certainly
    see segregating the .a static library, or even not shipping it at
    all, but what's it save to leave out the .so symlink?)
    Because the .so symlink is only needed at build time. At runtime, you
    need the .so.5.14 file. Hence .so.* -> $pkg, .h .a .so -> $pkg-dev.

    Christoph
    --
    cb@df7cb.de | http://www.df7cb.de/
  • Christian Ullrich at Jan 9, 2013 at 3:30 pm

    * Christoph Berg wrote:

    Re: Tom Lane 2013-01-09 <9802.1357702675@sss.pgh.pa.us>
    and Python.h. However, adding one won't fix your problem on
    Debian-based distros, because for some wacko reason they put the
    headers and the shlib .so symlink in different packages, cf
    http://packages.debian.org/squeeze/amd64/perl/filelist
    http://packages.debian.org/squeeze/amd64/libperl-dev/filelist
    I am unfamiliar with a good reason for doing that. (I can certainly
    see segregating the .a static library, or even not shipping it at
    all, but what's it save to leave out the .so symlink?)
    Because the .so symlink is only needed at build time. At runtime, you
    need the .so.5.14 file. Hence .so.* -> $pkg, .h .a .so -> $pkg-dev.
    That was Tom's point, I believe -- Debian does not do it that way.

    - perl-base has /usr/lib/libperl.so.5.etc
    - perl has the headers and a dependency on perl-base
    - libperl-dev has the symlink /usr/lib/libperl.so > libperl.so.5.etc

    So by installing "perl", you get enough to satisfy configure, but there
    is still no usable -lperl.

    --
    Christian
  • Tom Lane at Jan 9, 2013 at 4:00 pm

    Christian Ullrich writes:
    * Christoph Berg wrote:
    Re: Tom Lane 2013-01-09 <9802.1357702675@sss.pgh.pa.us>
    I am unfamiliar with a good reason for doing that. (I can certainly
    see segregating the .a static library, or even not shipping it at
    all, but what's it save to leave out the .so symlink?)
    Because the .so symlink is only needed at build time. At runtime, you
    need the .so.5.14 file. Hence .so.* -> $pkg, .h .a .so -> $pkg-dev.
    That was Tom's point, I believe -- Debian does not do it that way.
    - perl-base has /usr/lib/libperl.so.5.etc
    - perl has the headers and a dependency on perl-base
    - libperl-dev has the symlink /usr/lib/libperl.so > libperl.so.5.etc
    So by installing "perl", you get enough to satisfy configure, but there
    is still no usable -lperl.
    Right. [ dons red fedora for packaging purposes... ] The beef that
    I've got with this is that there is no sane reason to do it like that.
    If you are building C code against a library, you probably need both
    some headers and the .so symlink, neither of which will be needed at
    runtime; which is why both of those typically go into a foo-dev or
    foo-devel subpackage. There are some other situations involving dynamic
    loading where you might need the .so symlink at runtime, but in that
    case you typically end up deciding that the symlink had better be in the
    base package (Debian seems to have chosen this path for Python for
    instance).

    I'm not terribly familiar with building of Perl extensions, but it seems
    possible that there's some use for Perl's C headers in that process,
    which might explain why the headers are in "perl" and not a "perl-dev"
    subpackage. But since the symlink requires no space to speak of, the
    sensible thing to do with it would have been to include it in "perl"
    along with the headers.

    The libperl-dev package, as constituted, doesn't make any sense: it's
    got the symlink which people need, and a very large static (.a) library
    that most people don't need. Even worse, you can't tell without close
    inspection which of those files is actually used by a package that
    requires libperl-dev, and that is something that's important to know.
    Under Red Hat packaging rules, the .a library likely wouldn't be shipped
    at all; or if there was any demand for it, it would be all by its lonesome
    in a package named something like "libperl-static", so that it'd be easy
    to tell which packages actually use it. (Think about what happens if a
    security update needs to be made in that library ... how do you tell
    what has to be rebuilt? And in any case, discouraging use of the static
    library will reduce the number of packages to be rebuilt.)

    So IMO we're looking at some pretty broken packaging decisions here.
    I doubt we'll get Debian to change it, so I don't mind if someone
    wants to add a separate configure probe to verify libperl.so is
    available --- but as I said upthread, it's not going to be me, because
    I'm not in a position to test on the platforms that would be affected.

        regards, tom lane
  • Peter Eisentraut at Jan 9, 2013 at 5:07 pm

    On 1/9/13 11:00 AM, Tom Lane wrote:
    The libperl-dev package, as constituted, doesn't make any sense: it's
    got the symlink which people need, and a very large static (.a) library
    that most people don't need. Even worse, you can't tell without close
    inspection which of those files is actually used by a package that
    requires libperl-dev, and that is something that's important to know.
    The expectation is that if you want to link against libfoo, you install
    libfoo-dev, and after that you can uninstall it. What's wrong with that?
  • Tom Lane at Jan 9, 2013 at 5:50 pm

    Peter Eisentraut writes:
    On 1/9/13 11:00 AM, Tom Lane wrote:
    The libperl-dev package, as constituted, doesn't make any sense: it's
    got the symlink which people need, and a very large static (.a) library
    that most people don't need. Even worse, you can't tell without close
    inspection which of those files is actually used by a package that
    requires libperl-dev, and that is something that's important to know.
    The expectation is that if you want to link against libfoo, you install
    libfoo-dev, and after that you can uninstall it. What's wrong with that?
    What's wrong is that it's hard to tell whether the resulting package
    will contain a reference to the shared library (libperl.so.whatever)
    or an embedded copy of the static library. As I tried to explain, this
    is something that a well-run distro will want to be able to control,
    or at least determine automatically from the package's BuildRequires
    list (RPM-ism, not sure what Debian's package management stuff calls the
    equivalent concept). That makes it a bad idea independently of the
    problem of whether two configure tests are needed rather than one.

        regards, tom lane
  • Peter Eisentraut at Jan 10, 2013 at 9:18 pm

    On 1/9/13 12:50 PM, Tom Lane wrote:
    Peter Eisentraut <peter_e@gmx.net> writes:
    On 1/9/13 11:00 AM, Tom Lane wrote:
    The libperl-dev package, as constituted, doesn't make any sense: it's
    got the symlink which people need, and a very large static (.a) library
    that most people don't need. Even worse, you can't tell without close
    inspection which of those files is actually used by a package that
    requires libperl-dev, and that is something that's important to know.
    The expectation is that if you want to link against libfoo, you install
    libfoo-dev, and after that you can uninstall it. What's wrong with that?
    What's wrong is that it's hard to tell whether the resulting package
    will contain a reference to the shared library (libperl.so.whatever)
    or an embedded copy of the static library. As I tried to explain, this
    is something that a well-run distro will want to be able to control,
    or at least determine automatically from the package's BuildRequires
    list (RPM-ism, not sure what Debian's package management stuff calls the
    equivalent concept). That makes it a bad idea independently of the
    problem of whether two configure tests are needed rather than one.
    Yeah, this has nothing to do with how Perl is packaged. The issue of
    linking against static vs shared libraries is well understood, but a
    separate debate. There are other packaging tools that make sure this
    works out. Even if there were no static libraries, libperl-dev might
    still exist. The core issue here is that header files and link-time
    library files are in different packages that can be installed
    separately. Any other library were this choice was made would create
    the same issue. The fault is that configure checks only one thing and
    assumes two.
  • Tom Lane at Jan 10, 2013 at 9:38 pm

    Peter Eisentraut writes:
    ... The core issue here is that header files and link-time
    library files are in different packages that can be installed
    separately. Any other library were this choice was made would create
    the same issue. The fault is that configure checks only one thing and
    assumes two.
    It's complete folly to imagine that configure checks all and exactly the
    things that we require to build. In fact, I'd argue that a majority of
    what it does is inference or approximation of some sort. For an example
    we need look no further than the AC_CHECK_FUNC logic, which is not
    actually testing that a function-like symbol is available to be called.
    If autoconf were trying to be exact, its scripts would be many times
    larger and slower, but not many times more useful. Similarly, there is
    no very good reason for our configure code to spend cycles testing for
    cases that we don't expect to see in the field.

    In the case at hand, up to now we have been checking that you have
    ExtUtils::MakeMaker and assuming that if you do, you have perl headers
    and libperl.so. The first of these inferences seems valid enough to me,
    as the presence of MakeMaker suggests that you have facilities for
    building Perl extensions. As for the second, it's pretty common to
    suppose that headers and .so symlinks are packaged together.

    I continue to assert that Debian's choices here are less than sane.

    But in any case, the link test for libperl.so is going to go away again
    unless someone can explain how to make it work on Gentoo.

        regards, tom lane
  • Peter Eisentraut at Jan 10, 2013 at 10:07 pm

    On 1/10/13 4:38 PM, Tom Lane wrote:
    It's complete folly to imagine that configure checks all and exactly the
    things that we require to build. In fact, I'd argue that a majority of
    what it does is inference or approximation of some sort.
    I'm not saying they have to be exact. But you have to be prepared to
    make them more exact when your approximations don't work out in practice.
  • Peter Eisentraut at Jan 10, 2013 at 10:10 pm

    On 1/10/13 4:38 PM, Tom Lane wrote:
    The first of these inferences seems valid enough to me,
    as the presence of MakeMaker suggests that you have facilities for
    building Perl extensions.
    MakeMaker is perfectly useful without any C headers or C compiler nearby.
  • Peter Eisentraut at Jan 9, 2013 at 5:05 pm

    On 1/9/13 10:30 AM, Christian Ullrich wrote:
    * Christoph Berg wrote:
    Re: Tom Lane 2013-01-09 <9802.1357702675@sss.pgh.pa.us>
    and Python.h. However, adding one won't fix your problem on
    Debian-based distros, because for some wacko reason they put the
    headers and the shlib .so symlink in different packages, cf
    http://packages.debian.org/squeeze/amd64/perl/filelist
    http://packages.debian.org/squeeze/amd64/libperl-dev/filelist
    I am unfamiliar with a good reason for doing that. (I can certainly
    see segregating the .a static library, or even not shipping it at
    all, but what's it save to leave out the .so symlink?)
    Because the .so symlink is only needed at build time. At runtime, you
    need the .so.5.14 file. Hence .so.* -> $pkg, .h .a .so -> $pkg-dev.
    That was Tom's point, I believe -- Debian does not do it that way.

    - perl-base has /usr/lib/libperl.so.5.etc
    - perl has the headers and a dependency on perl-base
    - libperl-dev has the symlink /usr/lib/libperl.so > libperl.so.5.etc

    So by installing "perl", you get enough to satisfy configure, but there
    is still no usable -lperl.
    The reason it's like that is that the header files are usable for
    building Perl extensions, but that doesn't need the library. And the
    expectation is that if you want to build against libfoo, you install
    libfoo-dev. The fact that some other package also gives you half of
    what you need is a coincidence.

    The blame, if you want to assign any, is with configure, because it
    assumes that the header files and the library are an atomic unit and
    checks for the former and assumes the latter must be there as well.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouppgsql-hackers @
categoriespostgresql
postedJan 9, '13 at 1:48a
activeJan 14, '13 at 5:09p
posts39
users8
websitepostgresql.org...
irc#postgresql

People

Translate

site design / logo © 2021 Grokbase