While using PL/R in a web based application, I noticed that the library
load and initialization time is significant enough to be annoying. So I
wrote a quick hack to load and initialize the library on postmaster
startup. This way, the backends get a fully initialized copy of the
interpreter when they are forked. The hack was to postmaster.c just
after the SSL initialization code at about line 650 (just remembered
this is 7.3.2 though):

if (true) /* later use startup GUC var */
{
char *fullname = "$libdir/plr.so";
char *funcname = "start_interp";
func_ptr initfunc;

initfunc = (func_ptr)
load_external_function(fullname, funcname, true, NULL);
(*initfunc)();
}

(I also had to add a #define for func_ptr)

This brings me to a couple questions:

1. Is there anything inherently dangerous with this approach?
My light testing seems to show that it works quite well for
my purpose.

2. It seems to me that other libraries such as those for PL/Tcl,
PL/Perl, etc may have the same issue. Is there any merit in
a GUC variable to allow libraries such as this to be loaded
and initialized at postmaster start? I'll generalize this and
send in a patch if there is interest.

Joe

Search Discussions

  • Greg Stark at Feb 13, 2003 at 4:29 am

    Joe Conway writes:

    2. It seems to me that other libraries such as those for PL/Tcl,
    PL/Perl, etc may have the same issue. Is there any merit in
    a GUC variable to allow libraries such as this to be loaded
    and initialized at postmaster start? I'll generalize this and
    send in a patch if there is interest.
    A similar situation arises with mod_perl. Because perl is quite heavy-weight
    and systems often need lots of packages with static data it's common to load a
    startup.pl script that just loads lots of packages before the Apache server
    forks. This reduces memory usage drastically.

    The main gotcha is that you have to be careful about resources that you don't
    want shared. The typical case is database handles which are sockets that
    wouldn't be happy having two processes writing and reading on them.

    At first blush it seemed unlikely you would have a database connection in an
    embedded perl script. But then, hm, that would be a sly way of doing
    interdatabase connections. In any case there are other situations where you
    might want to have open file descriptors or sockets lying around.

    So in short, not only is it useful, but it would be valuable to allow
    mechanism to cause the language to load modules before forking. But there have
    to be prominent caveats that no such shared packages should create resources
    that can't be safely shared.

    --
    greg
  • Tom Lane at Feb 13, 2003 at 4:55 am

    Joe Conway writes:
    [ what about autoloading libraries into the postmaster? ]
    I can see a couple possible downsides: (a) the library might have some
    weird behavior across fork boundaries; (b) the additional memory space
    that has to be duplicated into child processes will cost something per
    child launch, even if the child never uses it. But these are only
    arguments that it might not *always* be a prudent thing to do, not that
    we shouldn't give the DBA the tool to do it if he wants. So fire away.

    (I seem to recall Peter muttering about linking plperl, pltcl, etc
    statically into the backend; which would reduce the need for this.
    But it would not eliminate it ... and he hasn't done it anyway...)

    regards, tom lane
  • Peter Eisentraut at Feb 13, 2003 at 9:15 pm

    Joe Conway writes:

    So I wrote a quick hack to load and initialize the library on postmaster
    startup.
    On glibc systems you can probably do this using the environment variable
    LD_PRELOAD. I guess others have a similar mechanism.

    --
    Peter Eisentraut peter_e@gmx.net
  • Joe Conway at Feb 13, 2003 at 9:25 pm

    Peter Eisentraut wrote:
    Joe Conway writes:
    So I wrote a quick hack to load and initialize the library on postmaster
    startup.
    On glibc systems you can probably do this using the environment variable
    LD_PRELOAD. I guess others have a similar mechanism.
    Hmmm. I could try that. But I found during testing that the loading was
    actually not the slow part, it was running the initialization function
    for the interpreter that was. I wonder if there is there any way to get
    an initialization function to automatically execute?

    Joe
  • Darko Prenosil at May 28, 2003 at 5:12 pm

    On Thursday 13 February 2003 22:24, Joe Conway wrote:
    Peter Eisentraut wrote:
    Joe Conway writes:
    So I wrote a quick hack to load and initialize the library on postmaster
    startup.
    On glibc systems you can probably do this using the environment variable
    LD_PRELOAD. I guess others have a similar mechanism.
    Hmmm. I could try that. But I found during testing that the loading was
    actually not the slow part, it was running the initialization function
    for the interpreter that was. I wonder if there is there any way to get
    an initialization function to automatically execute?
    Joe, did You got the answer to this question ?
    I would like to acomplish something like this:
    execute some stored procedure on backend(not postmaster) start and exit.
    So, it is not the same reason, but it is still the same question.

    Regards !

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouppgsql-hackers @
categoriespostgresql
postedFeb 13, '03 at 4:17a
activeMay 28, '03 at 5:12p
posts6
users5
websitepostgresql.org...
irc#postgresql

People

Translate

site design / logo © 2022 Grokbase