Tom,
My project, Veil, steals some of this shared memory for itself. I wan't
aware of the "slop factor" and was pleased that it just worked. I guess
I should have been asking questions of this group. Sorry.

I would like Veil to be a good citizen and place a legitimate request
for its shared memory (probably about 16K normally).

Veil is loaded as a shared library, which I would assume means that it
is not present during database startup, making contributing to the
sizing calculation and racing the lockmgr a little tricky.

I see a number of possibilities:

- Have a GUC to allocate shmem space for add-ons
- Have add-ons register themselves and provide hooks for sizing and
initial space allocation
- Let add-ons race with the lockmgr for the slop (ie leave as it is)

Thoughts?

I would be happy to work on this and provide whatever patches are
necessary.

Thanks.
__
Marc Munro

On Mon, 2005-10-17 at 10:34 -0300, pgsql-general-owner@postgresql.org
wrote:
Date: Mon, 17 Oct 2005 00:42:17 -0400
From: Tom Lane <tgl@sss.pgh.pa.us>
To: Douglas McNaught <doug@mcnaught.org>
Cc: cristian@clickdiario.com, tjo@acm.org,
"pgsql-general@postgresql.org" <pgsql-general@postgresql.org>
Subject: Re: dynamic loading of .so
Message-ID: <12614.1129524137@sss.pgh.pa.us>

Douglas McNaught <doug@mcnaught.org> writes:
<cristian@clickdiario.com> writes:
are there any way to make them global for all the instances?
Put them in shared memory. This probably isn't trival, as all the
shared memory is allocated up front and used for existing purposes (at
least, as I understand it).
There's a "slop factor" of 100KB or so in the shared memory size
calculation, which means that an add-on library that requests space
soon
enough could probably get away with allocating up to a few KB without
causing any problems. (The slop is not totally useless, since for
instance the lock manager might eat it up if more locks get requested
than expected.)

In the long run it might be interesting to add hooks that allow
preloaded libraries to contribute to the shmem sizing calculation and
then to snarf up the space they've requested before it can get eaten
by
the lockmgr.

regards, tom lane

Search Discussions

  • Bruce Momjian at Oct 24, 2005 at 1:24 pm
    Uh, why does Veil have to allocate space from the backend shared memory
    pool rather than allocating its own shared memory segment?

    ---------------------------------------------------------------------------

    Marc Munro wrote:
    -- Start of PGP signed section.
    Tom,
    My project, Veil, steals some of this shared memory for itself. I wan't
    aware of the "slop factor" and was pleased that it just worked. I guess
    I should have been asking questions of this group. Sorry.

    I would like Veil to be a good citizen and place a legitimate request
    for its shared memory (probably about 16K normally).

    Veil is loaded as a shared library, which I would assume means that it
    is not present during database startup, making contributing to the
    sizing calculation and racing the lockmgr a little tricky.

    I see a number of possibilities:

    - Have a GUC to allocate shmem space for add-ons
    - Have add-ons register themselves and provide hooks for sizing and
    initial space allocation
    - Let add-ons race with the lockmgr for the slop (ie leave as it is)

    Thoughts?

    I would be happy to work on this and provide whatever patches are
    necessary.

    Thanks.
    __
    Marc Munro

    On Mon, 2005-10-17 at 10:34 -0300, pgsql-general-owner@postgresql.org
    wrote:
    Date: Mon, 17 Oct 2005 00:42:17 -0400
    From: Tom Lane <tgl@sss.pgh.pa.us>
    To: Douglas McNaught <doug@mcnaught.org>
    Cc: cristian@clickdiario.com, tjo@acm.org,
    "pgsql-general@postgresql.org" <pgsql-general@postgresql.org>
    Subject: Re: dynamic loading of .so
    Message-ID: <12614.1129524137@sss.pgh.pa.us>

    Douglas McNaught <doug@mcnaught.org> writes:
    <cristian@clickdiario.com> writes:
    are there any way to make them global for all the instances?
    Put them in shared memory. This probably isn't trival, as all the
    shared memory is allocated up front and used for existing purposes (at
    least, as I understand it).
    There's a "slop factor" of 100KB or so in the shared memory size
    calculation, which means that an add-on library that requests space
    soon
    enough could probably get away with allocating up to a few KB without
    causing any problems. (The slop is not totally useless, since for
    instance the lock manager might eat it up if more locks get requested
    than expected.)

    In the long run it might be interesting to add hooks that allow
    preloaded libraries to contribute to the shmem sizing calculation and
    then to snarf up the space they've requested before it can get eaten
    by
    the lockmgr.

    regards, tom lane
    -- End of PGP section, PGP failed!

    --
    Bruce Momjian | http://candle.pha.pa.us
    pgman@candle.pha.pa.us | (610) 359-1001
    + If your life is a hard drive, | 13 Roberts Road
    + Christ can be your backup. | Newtown Square, Pennsylvania 19073
  • Marc Munro at Oct 24, 2005 at 4:16 pm
    Bruce,
    There are two problems, though maybe I came to the wrong solution. I'm
    not averse to changing it.

    1) Veil starts from a user process and not from the postmaster. This
    means that any shared memory segments created can not necessarily be
    mapped to the same address space in each process, which makes using such
    shared memory a little more painful than just referencing pointers.

    2) There is no simple way to close the shared memory segement when
    postgres shuts down.

    In an earlier version of Veil I did allocate my own shared memory and
    could revive the code if we can overcome problem number 2. I'd like to
    also overcome problem 1 as it makes the code extremely ugly but it's no
    show stopper.

    Any thoughts?

    __
    Marc
    On Mon, 2005-10-24 at 09:24 -0400, Bruce Momjian wrote:
    Uh, why does Veil have to allocate space from the backend shared memory
    pool rather than allocating its own shared memory segment?

    ---------------------------------------------------------------------------

    Marc Munro wrote:
    -- Start of PGP signed section.
    Tom,
    My project, Veil, steals some of this shared memory for itself. I wan't
    aware of the "slop factor" and was pleased that it just worked. I guess
    I should have been asking questions of this group. Sorry.

    I would like Veil to be a good citizen and place a legitimate request
    for its shared memory (probably about 16K normally).

    Veil is loaded as a shared library, which I would assume means that it
    is not present during database startup, making contributing to the
    sizing calculation and racing the lockmgr a little tricky.

    I see a number of possibilities:

    - Have a GUC to allocate shmem space for add-ons
    - Have add-ons register themselves and provide hooks for sizing and
    initial space allocation
    - Let add-ons race with the lockmgr for the slop (ie leave as it is)

    Thoughts?

    I would be happy to work on this and provide whatever patches are
    necessary.

    Thanks.
    __
    Marc Munro

    On Mon, 2005-10-17 at 10:34 -0300, pgsql-general-owner@postgresql.org
    wrote:
    Date: Mon, 17 Oct 2005 00:42:17 -0400
    From: Tom Lane <tgl@sss.pgh.pa.us>
    To: Douglas McNaught <doug@mcnaught.org>
    Cc: cristian@clickdiario.com, tjo@acm.org,
    "pgsql-general@postgresql.org" <pgsql-general@postgresql.org>
    Subject: Re: dynamic loading of .so
    Message-ID: <12614.1129524137@sss.pgh.pa.us>

    Douglas McNaught <doug@mcnaught.org> writes:
    <cristian@clickdiario.com> writes:
    are there any way to make them global for all the instances?
    Put them in shared memory. This probably isn't trival, as all the
    shared memory is allocated up front and used for existing purposes (at
    least, as I understand it).
    There's a "slop factor" of 100KB or so in the shared memory size
    calculation, which means that an add-on library that requests space
    soon
    enough could probably get away with allocating up to a few KB without
    causing any problems. (The slop is not totally useless, since for
    instance the lock manager might eat it up if more locks get requested
    than expected.)

    In the long run it might be interesting to add hooks that allow
    preloaded libraries to contribute to the shmem sizing calculation and
    then to snarf up the space they've requested before it can get eaten
    by
    the lockmgr.

    regards, tom lane
    -- End of PGP section, PGP failed!

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouppgsql-hackers @
categoriespostgresql
postedOct 17, '05 at 3:58p
activeOct 24, '05 at 4:16p
posts3
users2
websitepostgresql.org...
irc#postgresql

2 users in discussion

Marc Munro: 2 posts Bruce Momjian: 1 post

People

Translate

site design / logo © 2021 Grokbase