backend/libpq/pgcomm.c no longer compiles on my system. The cvs log sez

Massimo Dal Zotto <dz@cs.unitn.it>
socket-flock.patch
use advisory locks to check if the unix socket can be deleted.
A running postmaster keeps a lock on that file. A starting
postmaster exits if the file exists and is locked, otherwise
it deletes the sockets and proceeds.
This avoid the need to remove manually the file after a postmaster
or system crash.
I don't know if flock is available on any system. If not we could
define a HAVE_FLOCK set by configure.
flock is *VERY* far from portable. I am aware of three or four
different, mutually incompatible file locking syscalls on different
Unix flavors. flock is just one of the contestants. Even if the
call syntax were uniform, the semantics are not portable enough to
be safe (advisory locks don't work on NFS-mounted files, for example).

Massimo has a good idea in the long run, but I have strong doubts that
we want to start working the bugs out two days before a beta release
cycle. Portable file locking in Unix is a very nasty can of worms,
and I recommend not opening it at this particular point.

In short: I'd like to see this patch backed out until after 6.4.

regards, tom lane

Search Discussions

  • Tom at Aug 30, 1998 at 6:28 am

    On Sun, 30 Aug 1998, Tom Lane wrote:

    Massimo Dal Zotto <dz@cs.unitn.it>
    socket-flock.patch
    use advisory locks to check if the unix socket can be deleted.
    A running postmaster keeps a lock on that file. A starting
    postmaster exits if the file exists and is locked, otherwise
    it deletes the sockets and proceeds.
    This avoid the need to remove manually the file after a postmaster
    or system crash.
    I don't know if flock is available on any system. If not we could
    define a HAVE_FLOCK set by configure.
    flock is *VERY* far from portable. I am aware of three or four
    different, mutually incompatible file locking syscalls on different
    Unix flavors. flock is just one of the contestants. Even if the
    call syntax were uniform, the semantics are not portable enough to
    be safe (advisory locks don't work on NFS-mounted files, for example).
    You can't create unix domain sockets on NFS files systems (well, you
    might be allowed to, it just might not be very useful).

    The flock() call syntax is very consistant. The only other option is
    fcntl() which is very consistant too, and is described in POSIX.1

    The only other option is lockf(). No one uses this anymore. It was
    only ever supported on SVR2,3,4 anyhow. It also is just a subset of
    fcntl()
    Massimo has a good idea in the long run, but I have strong doubts that
    we want to start working the bugs out two days before a beta release
    cycle. Portable file locking in Unix is a very nasty can of worms,
    and I recommend not opening it at this particular point.
    Not really. The biggest problem is NFS issues, which don't need to be
    considered for unix domain sockets. The various sematics issues don't
    really apply for locking is being used for.
    In short: I'd like to see this patch backed out until after 6.4.

    regards, tom lane
    Tom
  • Massimo Dal Zotto at Aug 30, 1998 at 11:10 am

    backend/libpq/pgcomm.c no longer compiles on my system. The cvs log sez

    Massimo Dal Zotto <dz@cs.unitn.it>
    socket-flock.patch
    use advisory locks to check if the unix socket can be deleted.
    A running postmaster keeps a lock on that file. A starting
    postmaster exits if the file exists and is locked, otherwise
    it deletes the sockets and proceeds.
    This avoid the need to remove manually the file after a postmaster
    or system crash.
    I don't know if flock is available on any system. If not we could
    define a HAVE_FLOCK set by configure.
    flock is *VERY* far from portable. I am aware of three or four
    different, mutually incompatible file locking syscalls on different
    Unix flavors. flock is just one of the contestants. Even if the
    call syntax were uniform, the semantics are not portable enough to
    be safe (advisory locks don't work on NFS-mounted files, for example).

    Massimo has a good idea in the long run, but I have strong doubts that
    we want to start working the bugs out two days before a beta release
    cycle. Portable file locking in Unix is a very nasty can of worms,
    and I recommend not opening it at this particular point.

    In short: I'd like to see this patch backed out until after 6.4.
    Yes, I'm aware of this. For the moment I suggest we put a #ifdef linux
    around the code until a more portable solution is found.

    --
    Massimo Dal Zotto

    +----------------------------------------------------------------------+
    Massimo Dal Zotto email: dz@cs.unitn.it |
    Via Marconi, 141 phone: ++39-461-534251 |
    38057 Pergine Valsugana (TN) www: http://www.cs.unitn.it/~dz/ |
    Italy pgp: finger dz@tango.cs.unitn.it |
    +----------------------------------------------------------------------+
  • Bruce Momjian at Aug 30, 1998 at 12:24 pm
    [Charset iso-8859-1 unsupported, filtering to ASCII...]
    backend/libpq/pgcomm.c no longer compiles on my system. The cvs log sez

    Massimo Dal Zotto <dz@cs.unitn.it>
    socket-flock.patch
    use advisory locks to check if the unix socket can be deleted.
    A running postmaster keeps a lock on that file. A starting
    postmaster exits if the file exists and is locked, otherwise
    it deletes the sockets and proceeds.
    This avoid the need to remove manually the file after a postmaster
    or system crash.
    I don't know if flock is available on any system. If not we could
    define a HAVE_FLOCK set by configure.
    flock is *VERY* far from portable. I am aware of three or four
    different, mutually incompatible file locking syscalls on different
    Unix flavors. flock is just one of the contestants. Even if the
    call syntax were uniform, the semantics are not portable enough to
    be safe (advisory locks don't work on NFS-mounted files, for example).

    Massimo has a good idea in the long run, but I have strong doubts that
    we want to start working the bugs out two days before a beta release
    cycle. Portable file locking in Unix is a very nasty can of worms,
    and I recommend not opening it at this particular point.

    In short: I'd like to see this patch backed out until after 6.4.
    Yes, I'm aware of this. For the moment I suggest we put a #ifdef linux
    around the code until a more portable solution is found.

    Can't we just have configure check for flock(). Another idea is to
    create a 'pid' file in the pgsql/data/base directory, and do a kill -0
    to see if it is stil running before removing the lock.

    --
    Bruce Momjian | 830 Blythe Avenue
    maillist@candle.pha.pa.us | Drexel Hill, Pennsylvania 19026
    + If your life is a hard drive, | (610) 353-9879(w)
    + Christ can be your backup. | (610) 853-3000(h)
  • Tom Lane at Aug 30, 1998 at 3:25 pm

    Bruce Momjian writes:
    Can't we just have configure check for flock(). Another idea is to
    create a 'pid' file in the pgsql/data/base directory, and do a kill -0
    to see if it is stil running before removing the lock.
    The latter approach is what I was going to suggest. Writing a pid file
    would be a fine idea anyway --- for one thing, it makes it a lot easier
    to write a "kill the postmaster" script. Given that the postmaster
    should write a pid file, a new postmaster should look for an existing
    pid file, and try to do a kill(pid, 0) on the number contained therein.
    If this doesn't return an error, then you figure there is already a
    postmaster running, complain, and exit. Otherwise you figure you is it,
    (re)write the pid file and away you go. Then pqcomm.c can just
    unconditionally delete any old file that's in the way of making the
    pipe.

    The pidfile checking and creation probably ought to go in postmaster.c,
    not down inside pqcomm.c. I never liked the fact that a critical
    interlock function was being done by a low-level library that one might
    not even want to invoke (if all your clients are using TCP, opening up
    the Unix-domain socket is a waste of time, no?).

    BTW, there is another problem with relying on flock on the socket file
    for this purpose: it opens up a hole for a denial-of-service attack.
    Anyone who can write the file can flock it. (We already had a problem
    with DOS via creating a dummy file at /tmp/.s.PGSQL.5432, but it would
    be harder to spot the culprit with an flock-based interference.)

    regards, tom lane
  • Massimo Dal Zotto at Aug 30, 1998 at 4:23 pm

    Bruce Momjian <maillist@candle.pha.pa.us> writes:
    Can't we just have configure check for flock(). Another idea is to
    create a 'pid' file in the pgsql/data/base directory, and do a kill -0
    to see if it is stil running before removing the lock.
    The latter approach is what I was going to suggest. Writing a pid file
    would be a fine idea anyway --- for one thing, it makes it a lot easier
    to write a "kill the postmaster" script. Given that the postmaster
    should write a pid file, a new postmaster should look for an existing
    pid file, and try to do a kill(pid, 0) on the number contained therein.
    If this doesn't return an error, then you figure there is already a
    postmaster running, complain, and exit. Otherwise you figure you is it,
    (re)write the pid file and away you go. Then pqcomm.c can just
    unconditionally delete any old file that's in the way of making the
    pipe.

    The pidfile checking and creation probably ought to go in postmaster.c,
    not down inside pqcomm.c. I never liked the fact that a critical
    interlock function was being done by a low-level library that one might
    not even want to invoke (if all your clients are using TCP, opening up
    the Unix-domain socket is a waste of time, no?).

    BTW, there is another problem with relying on flock on the socket file
    for this purpose: it opens up a hole for a denial-of-service attack.
    Anyone who can write the file can flock it. (We already had a problem
    with DOS via creating a dummy file at /tmp/.s.PGSQL.5432, but it would
    be harder to spot the culprit with an flock-based interference.)
    This came to my mind, but I didn't think this would have happened so
    quickly. In my opinion the socket and the pidfile should be created in a
    directory owned by postgres, for example /tmp/.Pgsql-unix, like does X.

    --
    Massimo Dal Zotto

    +----------------------------------------------------------------------+
    Massimo Dal Zotto email: dz@cs.unitn.it |
    Via Marconi, 141 phone: ++39-461-534251 |
    38057 Pergine Valsugana (TN) www: http://www.cs.unitn.it/~dz/ |
    Italy pgp: finger dz@tango.cs.unitn.it |
    +----------------------------------------------------------------------+
  • Tom Lane at Aug 30, 1998 at 4:52 pm

    Massimo Dal Zotto writes:
    In my opinion the socket and the pidfile should be created in a
    directory owned by postgres, for example /tmp/.Pgsql-unix, like does X.
    The pidfile belongs at the top level of the database directory (eg,
    /usr/local/pgsql/data/postmaster.pid), because what it actually
    represents is that there is a postmaster running *for that database
    group*.

    If you want to support multiple database sets on one machine (which I
    do), then the interlock has to be per database directory. Putting the
    pidfile into a common directory would mean we'd have to invent some
    kind of pidfile naming convention to keep multiple postmasters from
    tromping on each other. This is unnecessarily complex.

    I agree with you that putting the socket file into a less easily munged
    directory than /tmp would be a good idea for security. But that's a
    separate issue. On machines that understand stickybits for directories,
    the security hole is not really very big.

    At this point, the fact that /tmp/.s.PGSQL.port# is the socket path is
    effectively a version-independent aspect of the FE/BE protocol, and so
    we can't change it without breaking old applications. I'm not sure that
    that's worth the security improvement.

    What I'd like to see someday is a postmaster command line switch to tell
    it to use *only* TCP connections and not create a Unix socket at all.
    That hasn't been possible so far, because we were relying on the socket
    file to provide a safety interlock against starting multiple
    postmasters. But an interlock using a pidfile would be much better.
    (Look around; *every* other Unix daemon I know of that wants to ensure
    that there's only one of it uses a pidfile interlock. Not file locking.
    There's a reason why that's the well-trodden path.)

    regards, tom lane
  • The Hermit Hacker at Aug 30, 1998 at 7:22 pm

    On Sun, 30 Aug 1998, Tom Lane wrote:

    Massimo Dal Zotto <dz@cs.unitn.it> writes:
    In my opinion the socket and the pidfile should be created in a
    directory owned by postgres, for example /tmp/.Pgsql-unix, like does X.
    The pidfile belongs at the top level of the database directory (eg,
    /usr/local/pgsql/data/postmaster.pid), because what it actually
    represents is that there is a postmaster running *for that database
    group*.
    I have to agree with this one...but then it also negates the
    argument about the flock() DoS...*grin*

    BTW...I like the kill(pid,0) solution myself, primarily because it
    is, i think, the most portable solution.

    I would not consider a patch to remove the flock() solution and
    replace it with the kill(pid,0) solution a new feature, just an
    improvement of an existing one...either way, moving the pid file (or
    socket, for that matter) from /tmp should be listed as a security related
    requirement for v6.4 :)

    Marc G. Fournier
    Systems Administrator @ hub.org
    primary: scrappy@hub.org secondary: scrappy@{freebsd|postgresql}.org
  • Tom Lane at Aug 31, 1998 at 2:34 am

    The Hermit Hacker writes:
    either way, moving the pid file (or
    socket, for that matter) from /tmp should be listed as a security related
    requirement for v6.4 :)
    Huh? There is no pid file being generated in /tmp (or anywhere else)
    at the moment. If we do add one, it should not go into /tmp for the
    reasons I gave before.

    Where the Unix-domain socket file lives is an entirely separate issue.

    If we move the socket out of /tmp then we have just kicked away all the
    work we did to preserve backwards compatibility of the FE/BE protocol
    with existing clients. Being able to talk to a 1.0 client isn't much
    good if you aren't listening where he's going to try to contact you.
    So I think I have to vote in favor of leaving the socket where it is.

    regards, tom lane
  • Bruce Momjian at Aug 31, 1998 at 4:37 am

    The Hermit Hacker writes:
    either way, moving the pid file (or
    socket, for that matter) from /tmp should be listed as a security related
    requirement for v6.4 :)
    Huh? There is no pid file being generated in /tmp (or anywhere else)
    at the moment. If we do add one, it should not go into /tmp for the
    reasons I gave before.

    Where the Unix-domain socket file lives is an entirely separate issue.

    If we move the socket out of /tmp then we have just kicked away all the
    work we did to preserve backwards compatibility of the FE/BE protocol
    with existing clients. Being able to talk to a 1.0 client isn't much
    good if you aren't listening where he's going to try to contact you.
    So I think I have to vote in favor of leaving the socket where it is.
    I have been thinking about this. First, we can easily use fopen(r+) to
    check to see if the file exists, and if it does read the pid and do a
    kill -0 to see if it is running. If no one else does it, I will take it
    on.

    Second, where to put the pid file. There is reason to put in /tmp,
    because it will get cleared in a reboot, and because it is locking the
    port number 5432. There is also reason to put it in /data because you
    can't have more than one postmaster running on a single data directory.

    So, we really want to lock both places. If this is going to make it
    easier for people to run more than one postmaster, because it will
    prevent/warn administrators when they try and put two postmasters in the
    same data dir or port, I say create the pid lock files both places, and
    give the admin a clear description of what he is doing wrong in each
    case.


    --
    Bruce Momjian | 830 Blythe Avenue
    maillist@candle.pha.pa.us | Drexel Hill, Pennsylvania 19026
    + If your life is a hard drive, | (610) 353-9879(w)
    + Christ can be your backup. | (610) 853-3000(h)
  • David Gould at Aug 31, 1998 at 8:23 am

    The Hermit Hacker <scrappy@hub.org> writes:
    either way, moving the pid file (or
    socket, for that matter) from /tmp should be listed as a security related
    requirement for v6.4 :)
    Huh? There is no pid file being generated in /tmp (or anywhere else)
    at the moment. If we do add one, it should not go into /tmp for the
    reasons I gave before.

    Where the Unix-domain socket file lives is an entirely separate issue.

    If we move the socket out of /tmp then we have just kicked away all the
    work we did to preserve backwards compatibility of the FE/BE protocol
    with existing clients. Being able to talk to a 1.0 client isn't much
    good if you aren't listening where he's going to try to contact you.
    So I think I have to vote in favor of leaving the socket where it is.
    I have been thinking about this. First, we can easily use fopen(r+) to
    check to see if the file exists, and if it does read the pid and do a
    kill -0 to see if it is running. If no one else does it, I will take it
    on.

    Second, where to put the pid file. There is reason to put in /tmp,
    because it will get cleared in a reboot, and because it is locking the
    port number 5432. There is also reason to put it in /data because you
    can't have more than one postmaster running on a single data directory.

    So, we really want to lock both places. If this is going to make it
    easier for people to run more than one postmaster, because it will
    prevent/warn administrators when they try and put two postmasters in the
    same data dir or port, I say create the pid lock files both places, and
    give the admin a clear description of what he is doing wrong in each
    case.

    If the traffic on bugtraq is any indication, writing in /tmp is a
    security exposure for daemons. Typical attack is:

    ln -s /etc/passwd /tmp/dumbrootprog.tmpfile

    when dumprootprog runs it writes on /etc/passwd. Cool huh? Not so serious
    in our case, as only pgsql owned files are at risk, but why take a chance.
    This argues for the private pid file.

    Also, before sprinkling files all over it is good to try to conform
    to the FHS (File Hierarchy Standard) (see http://www.pathname.com/fhs/)
    which is pretty easy to do and likely to make life easier later.

    -dg

    David Gould dg@informix.com 510.628.3783 or 510.305.9468
    Informix Software (No, really) 300 Lakeside Drive Oakland, CA 94612
    - If simplicity worked, the world would be overrun with insects. -
  • The Hermit Hacker at Aug 31, 1998 at 12:24 pm

    On Mon, 31 Aug 1998, David Gould wrote:

    Also, before sprinkling files all over it is good to try to conform
    to the FHS (File Hierarchy Standard) (see http://www.pathname.com/fhs/)
    which is pretty easy to do and likely to make life easier later.
    I just downloaded and skim'd the fhs notes, and it looks
    reasonable...but, other then the current socket in /tmp, we don't
    "splinkle files all over"...do we? *raised eyebrow*
  • David Gould at Aug 31, 1998 at 5:50 pm

    On Mon, 31 Aug 1998, David Gould wrote:

    Also, before sprinkling files all over it is good to try to conform
    to the FHS (File Hierarchy Standard) (see http://www.pathname.com/fhs/)
    which is pretty easy to do and likely to make life easier later.
    I just downloaded and skim'd the fhs notes, and it looks
    reasonable...but, other then the current socket in /tmp, we don't
    "splinkle files all over"...do we? *raised eyebrow*
    Well, perhaps not, thay might have been hyperbole, but I wanted to head it
    off before it happened.

    Also, what about moving the socket to the PG_DATA dir and then creating a
    symlink to it in /tmp for older clients. New installations could optionally
    not create (or remove) the symlink...

    -dg

    David Gould dg@informix.com 510.628.3783 or 510.305.9468
    Informix Software (No, really) 300 Lakeside Drive Oakland, CA 94612
    - If simplicity worked, the world would be overrun with insects. -
  • Bruce Momjian at Aug 31, 1998 at 6:32 pm

    On Mon, 31 Aug 1998, David Gould wrote:

    Also, before sprinkling files all over it is good to try to conform
    to the FHS (File Hierarchy Standard) (see http://www.pathname.com/fhs/)
    which is pretty easy to do and likely to make life easier later.
    I just downloaded and skim'd the fhs notes, and it looks
    reasonable...but, other then the current socket in /tmp, we don't
    "splinkle files all over"...do we? *raised eyebrow*
    Well, perhaps not, thay might have been hyperbole, but I wanted to head it
    off before it happened.

    Also, what about moving the socket to the PG_DATA dir and then creating a
    symlink to it in /tmp for older clients. New installations could optionally
    not create (or remove) the symlink...
    But then we have to compile the data directory into the client, or add
    an option to specify the unix domain socket AND the data directory. Not
    worth it, I think.

    --
    Bruce Momjian | 830 Blythe Avenue
    maillist@candle.pha.pa.us | Drexel Hill, Pennsylvania 19026
    + If your life is a hard drive, | (610) 353-9879(w)
    + Christ can be your backup. | (610) 853-3000(h)
  • Massimo Dal Zotto at Aug 31, 1998 at 9:05 pm

    On Mon, 31 Aug 1998, David Gould wrote:

    Also, before sprinkling files all over it is good to try to conform
    to the FHS (File Hierarchy Standard) (see http://www.pathname.com/fhs/)
    which is pretty easy to do and likely to make life easier later.
    I just downloaded and skim'd the fhs notes, and it looks
    reasonable...but, other then the current socket in /tmp, we don't
    "splinkle files all over"...do we? *raised eyebrow*
    Well, perhaps not, thay might have been hyperbole, but I wanted to head it
    off before it happened.

    Also, what about moving the socket to the PG_DATA dir and then creating a
    symlink to it in /tmp for older clients. New installations could optionally
    not create (or remove) the symlink...

    -dg

    David Gould dg@informix.com 510.628.3783 or 510.305.9468
    Informix Software (No, really) 300 Lakeside Drive Oakland, CA 94612
    - If simplicity worked, the world would be overrun with insects. -
    In the fhs notes I read:


    ------------------------------------------------------------------------
    5.9 /var/run : Run-time variable files

    This directory contains system information files describing the system
    since it was booted. Files in this directory should be cleared (removed
    or truncated as appropriate) at the beginning of the boot process.

    Process identifier (PID) files, which were originally placed in /etc,
    should be placed in /var/run. The naming convention for PID files is
    <program-name>.pid. For example, the crond PID file is named
    /var/run/crond.pid.

    The internal format of PID files remains unchanged. The file should
    consist of the process identifier in ASCII-encoded decimal, followed by
    a newline character. For example, if crond was process number 25,
    /var/run/crond.pid would contain three characters: two, five, and
    newline.

    Programs that read PID files should be somewhat flexible in what they
    accept; i.e., they should ignore extra whitespace, leading zeroes,
    absence of the trailing newline, or additional lines in the PID file.
    Programs that create PID files should use the simple specification
    located in the above paragraph.

    The utmp file, which stores information about who is currently using the
    system, is located in this directory.

    Programs that maintain transient UNIX-domain sockets should place them
    in this directory.
    ------------------------------------------------------------------------


    It seems that if we want to follow this document there is no much to
    discuss. But on my linux installation this directory is owned by root
    so I don't see how pgsql could create a socket in it. What standards
    are in use on other systems ?

    --
    Massimo Dal Zotto

    +----------------------------------------------------------------------+
    Massimo Dal Zotto email: dz@cs.unitn.it |
    Via Marconi, 141 phone: ++39-461-534251 |
    38057 Pergine Valsugana (TN) www: http://www.cs.unitn.it/~dz/ |
    Italy pgp: finger dz@tango.cs.unitn.it |
    +----------------------------------------------------------------------+
  • Thomas G. Lockhart at Sep 1, 1998 at 1:31 am

    Also, what about moving the socket to the PG_DATA dir and then
    creating a symlink to it in /tmp for older clients.
    Clients don't have visibility into $PG_DATA, do they? Just ran into this
    working on the ODBC interface, trying to find a place for a system-wide
    configuration file. Ended up putting it in $POSTGRESDIR by default.

    The /var/run option (or something similar) seems to be a good way to
    head, if we can get enough support on the different platforms. Actually,
    this could be an autoconf test, couldn't it?

    - Thomas
  • Tom Lane at Aug 31, 1998 at 2:20 pm

    David Gould writes:
    If the traffic on bugtraq is any indication, writing in /tmp is a
    security exposure for daemons. Typical attack is:
    ln -s /etc/passwd /tmp/dumbrootprog.tmpfile
    A partial answer to this is to unlink /tmp/pidfile before trying to
    create/write it. There's still a window for the attack, but it's
    mighty tiny. (You do have to check that the unlink either succeeds
    or fails with ENOENT --- in particular, an EPERM failure is trouble
    for obvious reasons.)

    I finally understand where Bruce is coming from on this point: he wants
    pid lock files in *both* the data dir (to lock the database) and /tmp
    (to lock the Unix-socket port number). This makes sense to me, and I
    agree that it'd become a lot safer to run multiple postmasters/databases
    that way. The data and the port are independent resources and each one
    needs a lock.

    I just came up with an idea that might help alleviate the /tmp security
    exposure without creating a backwards-compatibility problem. It works
    like this:

    1. During installation, create a subdirectory of /tmp to hold Postgres'
    socket files and associated pid lockfiles. This subdirectory should be
    owned by the Postgres superuser and have permissions 755
    (world-readable, writable only by Postgres superuser). Maybe call it
    /tmp/.pgsql --- the name should start with a dot to keep it out of the
    way. (Bruce points out that some systems clear /tmp during reboot, so
    it might be that a postmaster will have to be prepared to recreate this
    directory at startup --- anyone know if subdirectories of /tmp are
    zapped too? My system doesn't do that...)

    2. When a postmaster fires up, it checks for/creates a pid lockfile in
    the subdirectory, and also creates the socket file in the subdirectory.

    3. For backwards compatibility with 1.0 clients, the socket file is also
    hard-linked to /tmp/.s.PGSQL.port# (use an unlink() and link()).

    This way, there's no security risk of overwriting someone else's file,
    since we never create or write on a file in a directory we don't own, we
    only try to link an already-existing socket file into /tmp.

    I'd suggest that the pid and socket files be given names in /tmp/.pgsql
    that don't start with dots --- no need to make them hard to see in that
    directory.

    We can change libpq to find the socket at /tmp/.pgsql/whatever, and
    eventually we could perhaps drop the backwards-compatibility feature
    of creating a link in /tmp.
    Also, before sprinkling files all over it is good to try to conform
    to the FHS (File Hierarchy Standard) (see http://www.pathname.com/fhs/)
    which is pretty easy to do and likely to make life easier later.
    I notice that on my system, the X11 socket files in /tmp/.X11-unix are
    actually symlinks to socket files in /usr/spool/sockets/X11. I dunno if
    it's worth our trouble to get into putting our sockets under /usr/spool
    or /var/spool or whatever --- seems like another configuration choice to
    mess up. It'd be nice if the socket directory lived somewhere where the
    parent dirs weren't world-writable, but this would mean one more thing
    that you have to have root permissions for in order to install pgsql.

    regards, tom lane
  • Brook Milligan at Aug 31, 1998 at 3:10 pm
    I just came up with an idea that might help alleviate the /tmp security
    exposure without creating a backwards-compatibility problem. It works
    like this:

    1. During installation, create a subdirectory of /tmp to hold Postgres'
    socket files and associated pid lockfiles. This subdirectory should be
    owned by the Postgres superuser and have permissions 755
    (world-readable, writable only by Postgres superuser). Maybe call it
    /tmp/.pgsql --- the name should start with a dot to keep it out of the
    way. (Bruce points out that some systems clear /tmp during reboot, so
    it might be that a postmaster will have to be prepared to recreate this
    directory at startup --- anyone know if subdirectories of /tmp are
    zapped too? My system doesn't do that...)

    ...

    I notice that on my system, the X11 socket files in /tmp/.X11-unix are
    actually symlinks to socket files in /usr/spool/sockets/X11. I dunno if
    it's worth our trouble to get into putting our sockets under /usr/spool
    or /var/spool or whatever --- seems like another configuration choice to
    mess up. It'd be nice if the socket directory lived somewhere where the
    parent dirs weren't world-writable, but this would mean one more thing
    that you have to have root permissions for in order to install pgsql.

    It seems like we need a directory for locks (= pid files) and one for
    sockets (perhaps the same one). I strongly suggest that the location
    for these be configurable. By default, it might make sense to put
    them in ~pgsql/locks and ~pgsql/sockets. It is easy (i.e., I'll be
    glad to do it) to modify configure.in to take options like

    --lock-dir=/var/spool/lock
    --socket-dir=/var/spool/sockets

    that set cc defines and have the code respond accordingly. This way,
    those who don't care (or don't have root access) can use the defaults,
    whereas those with root access who like to keep locks and sockets in a
    common place can do so easily. Either way, multiple postmasters (all
    compiled with the same options of course) can check the appropriate
    locks in the well-known places. Finally, drop the link into /tmp for
    the old socket and document that it will be disappearing at some
    point, and all is fine.

    If someone wants to give me some guidance on what preprocessor
    variables should be set in response to the above options (or something
    like them), I'll do the configure stuff.

    Cheers,
    Brook
  • Bruce Momjian at Aug 31, 1998 at 5:04 pm

    I just came up with an idea that might help alleviate the /tmp security
    exposure without creating a backwards-compatibility problem. It works
    like this:

    1. During installation, create a subdirectory of /tmp to hold Postgres'
    socket files and associated pid lockfiles. This subdirectory should be
    owned by the Postgres superuser and have permissions 755
    (world-readable, writable only by Postgres superuser). Maybe call it
    /tmp/.pgsql --- the name should start with a dot to keep it out of the
    way. (Bruce points out that some systems clear /tmp during reboot, so
    it might be that a postmaster will have to be prepared to recreate this
    directory at startup --- anyone know if subdirectories of /tmp are
    zapped too? My system doesn't do that...)

    ...

    I notice that on my system, the X11 socket files in /tmp/.X11-unix are
    actually symlinks to socket files in /usr/spool/sockets/X11. I dunno if
    it's worth our trouble to get into putting our sockets under /usr/spool
    or /var/spool or whatever --- seems like another configuration choice to
    mess up. It'd be nice if the socket directory lived somewhere where the
    parent dirs weren't world-writable, but this would mean one more thing
    that you have to have root permissions for in order to install pgsql.

    It seems like we need a directory for locks (= pid files) and one for
    sockets (perhaps the same one). I strongly suggest that the location
    for these be configurable. By default, it might make sense to put
    them in ~pgsql/locks and ~pgsql/sockets. It is easy (i.e., I'll be
    glad to do it) to modify configure.in to take options like

    --lock-dir=/var/spool/lock
    --socket-dir=/var/spool/sockets

    that set cc defines and have the code respond accordingly. This way,
    those who don't care (or don't have root access) can use the defaults,
    whereas those with root access who like to keep locks and sockets in a
    common place can do so easily. Either way, multiple postmasters (all
    compiled with the same options of course) can check the appropriate
    locks in the well-known places. Finally, drop the link into /tmp for
    the old socket and document that it will be disappearing at some
    point, and all is fine.

    If someone wants to give me some guidance on what preprocessor
    variables should be set in response to the above options (or something
    like them), I'll do the configure stuff.
    I think we need to keep stuff in /tmp. No reason to add more
    configuration just for the sake of it.

    If Tom says we can use symbolic links to sockets, why not just do that
    for backward compatability.

    On my system, all of /tmp is wiped out on reboot because it is a memory
    file system.

    Let each postmaster start up and create it's own directory as
    /tmp/.pgsql.5432 or whatever port number they use. In the directory, it
    can put its socket and pid file. We also put a pid file in the
    postmaster's data directory, and give admins errors if they try anything
    funny.

    I don't think we can use a main /tmp/.pgsql directory because one system
    can have multiple postmasters run by different users. If we start
    putting it in /usr/local/pgsql, we need the socket in a common place,
    because the actual pgsql directory can be anywhere, so we need /tmp
    anyway. We can't have clients being configured for different pgsql
    locations.

    We still have a denial of service attach if someone creates a
    .pgsql.5432 directory before we do, but the unix domain socket has to be
    in a common place, and we can't just kick people out common territory
    because we are not root.

    Again, we could put it all in pgsql, but we still need that unix domain
    socket, and that can't be configured into the client easily, so I think
    we just have to live with /tmp.

    Even if we put all the sockets in /tmp/.pgsql, we have the problem of
    someone creating /tmp/.pgsql before we start, or after a reboot. No way
    around it that I can see.

    In fact, I can't really see a reason for the separate directory in /tmp
    vs. what we do now, except that we assume /tmp has the sticky bit, while
    creation of our own directory with our own permissions fixes that, which
    may be a win, especially because X and others do it that way.

    --
    Bruce Momjian | 830 Blythe Avenue
    maillist@candle.pha.pa.us | Drexel Hill, Pennsylvania 19026
    + If your life is a hard drive, | (610) 353-9879(w)
    + Christ can be your backup. | (610) 853-3000(h)
  • Bruce Momjian at Aug 31, 1998 at 5:10 pm

    dg@informix.com (David Gould) writes:
    If the traffic on bugtraq is any indication, writing in /tmp is a
    security exposure for daemons. Typical attack is:
    ln -s /etc/passwd /tmp/dumbrootprog.tmpfile
    A partial answer to this is to unlink /tmp/pidfile before trying to
    create/write it. There's still a window for the attack, but it's
    mighty tiny. (You do have to check that the unlink either succeeds
    or fails with ENOENT --- in particular, an EPERM failure is trouble
    for obvious reasons.)
    The real fix for this is to do an open(O_CREAT) to force creation at the
    time of the open. If it fails, try removing it and try again.

    With our own directory, it is even easier. If no pid locks,
    unconditionally drop the directory, and recreate it with our
    permissions, and if the create fails, try again. We don't continue with
    the postmaster until we drop and create the file or directory. Can't
    symlink hack around that because we force creation.

    1. During installation, create a subdirectory of /tmp to hold Postgres'
    socket files and associated pid lockfiles. This subdirectory should be
    owned by the Postgres superuser and have permissions 755
    (world-readable, writable only by Postgres superuser). Maybe call it
    /tmp/.pgsql --- the name should start with a dot to keep it out of the
    Problem is multiple postmasters with different users.
    way. (Bruce points out that some systems clear /tmp during reboot, so
    it might be that a postmaster will have to be prepared to recreate this
    directory at startup --- anyone know if subdirectories of /tmp are
    zapped too? My system doesn't do that...)
    Get's wiped out. Has to be done only by the postmaster. Assuming
    initdb is going to create something that is going to stay around in /tmp
    is going to break.
    2. When a postmaster fires up, it checks for/creates a pid lockfile in
    the subdirectory, and also creates the socket file in the subdirectory.
    Forces creation.
    3. For backwards compatibility with 1.0 clients, the socket file is also
    hard-linked to /tmp/.s.PGSQL.port# (use an unlink() and link()).
    Good idea.
    This way, there's no security risk of overwriting someone else's file,
    since we never create or write on a file in a directory we don't own, we
    only try to link an already-existing socket file into /tmp.

    I'd suggest that the pid and socket files be given names in /tmp/.pgsql
    that don't start with dots --- no need to make them hard to see in that
    directory.
    Yes, stuff in the directory doesn't need dots.
    We can change libpq to find the socket at /tmp/.pgsql/whatever, and
    eventually we could perhaps drop the backwards-compatibility feature
    of creating a link in /tmp.
    OK
    Also, before sprinkling files all over it is good to try to conform
    to the FHS (File Hierarchy Standard) (see http://www.pathname.com/fhs/)
    which is pretty easy to do and likely to make life easier later.
    I notice that on my system, the X11 socket files in /tmp/.X11-unix are
    actually symlinks to socket files in /usr/spool/sockets/X11. I dunno if
    it's worth our trouble to get into putting our sockets under /usr/spool
    or /var/spool or whatever --- seems like another configuration choice to
    mess up. It'd be nice if the socket directory lived somewhere where the
    parent dirs weren't world-writable, but this would mean one more thing
    that you have to have root permissions for in order to install pgsql.
    Yes, too compilicated.

    --
    Bruce Momjian | 830 Blythe Avenue
    maillist@candle.pha.pa.us | Drexel Hill, Pennsylvania 19026
    + If your life is a hard drive, | (610) 353-9879(w)
    + Christ can be your backup. | (610) 853-3000(h)
  • Billy G. Allie at Sep 8, 1998 at 3:02 pm

    Bruce Momjian writes:

    I have been thinking about this. First, we can easily use fopen(r+) to
    check to see if the file exists, and if it does read the pid and do a
    kill -0 to see if it is running. If no one else does it, I will take it
    on.
    It is better to use open with the O_CREAT and O_EXCL set. If the file does not
    exist it will be created and the PID can be written to it. If the file exists
    then the call will fail, at which point it can be opened with fread, and the
    PID it contains can be checked to see if it still exists with kill. The open
    call has the added advantage that 'The check for the existence of the file and
    the creation of the file if it does not exist is atomic with respect to other
    processes executing open naming the same filename in the same directory with
    O_EXCL and O_CREAT set.' [from the UnixAWare 7 man page, open(2)].

    Also, you can't just delete the file, create it and write the your PID to it
    and assume that you have the lock, you need to close the file, sleep some
    small amount of time and then open and read the file to see if you still have
    the lock. If you like, I can take this task on.

    Oh, the postmaster must clear the PID when it exits.
    Second, where to put the pid file. There is reason to put in /tmp,
    because it will get cleared in a reboot, and because it is locking the
    port number 5432. There is also reason to put it in /data because you
    can't have more than one postmaster running on a single data directory.

    So, we really want to lock both places. If this is going to make it
    easier for people to run more than one postmaster, because it will
    prevent/warn administrators when they try and put two postmasters in the
    same data dir or port, I say create the pid lock files both places, and
    give the admin a clear description of what he is doing wrong in each
    case.
    IHMO, the pid should be put in the data directory. The reasoning that it will get cleared in a reboot is not sufficent since the logic used to create the PID file will delete it if the PID it contains is not a running process. Besides, I have used systems where /tmp was not cleared out on a re-boot (for various reasons). Also, I would rather have a script that explicitly removes the PID locking file at system statup (if it exists), in which case, it doesn't matter where it resides.
    --
    ____ | Billy G. Allie | Domain....: Bill.Allie@mug.org
    /| | 7436 Hartwell | Compuserve: 76337,2061
    -/-|----- | Dearborn, MI 48126| MSN.......: B_G_Allie@email.msn.com
    / |LLIE | (313) 582-1540 |
  • The Hermit Hacker at Oct 30, 1998 at 2:11 am

    On Sun, 30 Aug 1998, Tom Lane wrote:

    The Hermit Hacker <scrappy@hub.org> writes:
    either way, moving the pid file (or
    socket, for that matter) from /tmp should be listed as a security related
    requirement for v6.4 :)
    Huh? There is no pid file being generated in /tmp (or anywhere else)
    at the moment. If we do add one, it should not go into /tmp for the
    reasons I gave before.

    Where the Unix-domain socket file lives is an entirely separate issue.

    If we move the socket out of /tmp then we have just kicked away all the
    work we did to preserve backwards compatibility of the FE/BE protocol
    with existing clients. Being able to talk to a 1.0 client isn't much
    good if you aren't listening where he's going to try to contact you.
    So I think I have to vote in favor of leaving the socket where it is.
    Let me put my vote in...: $PGSQLHOME/run/{pgsql.pid,pgsql.pid.socket}

    Screw backwards compatibility...v6.5 becomes v7.0 *shrug*

    Marc G. Fournier
    Systems Administrator @ hub.org
    primary: scrappy@hub.org secondary: scrappy@{freebsd|postgresql}.org
  • Tom Lane at Oct 30, 1998 at 4:18 am

    The Hermit Hacker writes:
    On Sun, 30 Aug 1998, Tom Lane wrote:
    So I think I have to vote in favor of leaving the socket where it is.
    Let me put my vote in...: $PGSQLHOME/run/{pgsql.pid,pgsql.pid.socket}
    Screw backwards compatibility...v6.5 becomes v7.0 *shrug*
    Catching up on back email, Marc?

    I think the end consensus of that thread was that we should move the
    socket file to someplace safer than /tmp (exactly where being a
    configure-time choice), *and* optionally put a softlink to it in /tmp
    for backwards compatibility with old clients. Use of the /tmp link
    would be deprecated because of possible security risks, but it could
    continue to work for as long as a particular installation needed to
    keep that option enabled.

    Actually getting it done doesn't seem to have happened :-(. I think
    we were all viewing this as part & parcel of fixing the lockfile issues,
    which no one got exercised enough to do. Too many higher-priority
    tasks...

    regards, tom lane
  • The Hermit Hacker at Oct 30, 1998 at 4:24 am

    On Thu, 29 Oct 1998, Tom Lane wrote:

    The Hermit Hacker <scrappy@hub.org> writes:
    On Sun, 30 Aug 1998, Tom Lane wrote:
    So I think I have to vote in favor of leaving the socket where it is.
    Let me put my vote in...: $PGSQLHOME/run/{pgsql.pid,pgsql.pid.socket}
    Screw backwards compatibility...v6.5 becomes v7.0 *shrug*
    Catching up on back email, Marc?
    I have a cold and am partially delirious? :) The cold part is
    accurate, at least :)

    Marc G. Fournier
    Systems Administrator @ hub.org
    primary: scrappy@hub.org secondary: scrappy@{freebsd|postgresql}.org
  • Bruce Momjian at Oct 30, 1998 at 4:31 pm

    The Hermit Hacker writes:
    On Sun, 30 Aug 1998, Tom Lane wrote:
    So I think I have to vote in favor of leaving the socket where it is.
    Let me put my vote in...: $PGSQLHOME/run/{pgsql.pid,pgsql.pid.socket}
    Screw backwards compatibility...v6.5 becomes v7.0 *shrug*
    Catching up on back email, Marc?

    I think the end consensus of that thread was that we should move the
    socket file to someplace safer than /tmp (exactly where being a
    configure-time choice), *and* optionally put a softlink to it in /tmp
    for backwards compatibility with old clients. Use of the /tmp link
    would be deprecated because of possible security risks, but it could
    continue to work for as long as a particular installation needed to
    keep that option enabled.
    Just a reminder that if you move it, all clients must know the new
    location, or somehow the location must be accessable by the clients.


    --
    Bruce Momjian | http://www.op.net/~candle
    maillist@candle.pha.pa.us | (610) 853-3000
    + If your life is a hard drive, | 830 Blythe Avenue
    + Christ can be your backup. | Drexel Hill, Pennsylvania 19026
  • Tom Lane at Oct 30, 1998 at 4:44 pm

    Bruce Momjian writes:
    I think the end consensus of that thread was that we should move the
    socket file to someplace safer than /tmp (exactly where being a
    configure-time choice), *and* optionally put a softlink to it in /tmp
    for backwards compatibility with old clients.
    Just a reminder that if you move it, all clients must know the new
    location, or somehow the location must be accessable by the clients.
    That's why I said *configure* time choice. The socket location has to
    be compiled into both frontends and backends. But there's no reason
    why the location couldn't be set during site configure, rather than
    being hard-wired as /tmp/.s.PGSQL.####.

    Supporting a softlink in /tmp would be helpful if an installation
    doesn't want to rebuild all their existing frontend apps right away
    (or at least all the ones that are on the server's local machine and use
    Unix-socket connections). Fortunately, cross-machine connections don't
    use the socket file, or it wouldn't really be practical to make the
    socket location site-dependent.

    regards, tom lane
  • The Hermit Hacker at Oct 30, 1998 at 8:20 pm

    On Fri, 30 Oct 1998, Bruce Momjian wrote:

    The Hermit Hacker <scrappy@hub.org> writes:
    On Sun, 30 Aug 1998, Tom Lane wrote:
    So I think I have to vote in favor of leaving the socket where it is.
    Let me put my vote in...: $PGSQLHOME/run/{pgsql.pid,pgsql.pid.socket}
    Screw backwards compatibility...v6.5 becomes v7.0 *shrug*
    Catching up on back email, Marc?

    I think the end consensus of that thread was that we should move the
    socket file to someplace safer than /tmp (exactly where being a
    configure-time choice), *and* optionally put a softlink to it in /tmp
    for backwards compatibility with old clients. Use of the /tmp link
    would be deprecated because of possible security risks, but it could
    continue to work for as long as a particular installation needed to
    keep that option enabled.
    Just a reminder that if you move it, all clients must know the new
    location, or somehow the location must be accessable by the clients.
    Which is only relevant if using Unix-domain sockets...

    Marc G. Fournier
    Systems Administrator @ hub.org
    primary: scrappy@hub.org secondary: scrappy@{freebsd|postgresql}.org
  • Bruce Momjian at Oct 2, 1998 at 6:32 pm
    Massimo, do we have all the documentation change for the new features
    you added. I am thinking particularly about the new -d debug levels,
    but there may be more.



    --
    Bruce Momjian | http://www.op.net/~candle
    maillist@candle.pha.pa.us | (610) 853-3000
    + If your life is a hard drive, | 830 Blythe Avenue
    + Christ can be your backup. | Drexel Hill, Pennsylvania 19026

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouppgsql-hackers @
categoriespostgresql
postedAug 30, '98 at 5:42a
activeOct 30, '98 at 8:20p
posts28
users9
websitepostgresql.org...
irc#postgresql

People

Translate

site design / logo © 2022 Grokbase