[PostgreSQL-Hackers] Re: [PATCHES] patches for 6.2.1p6

Hi hackers,

I have old patches for version 6.2.1p6 which fix some problems and add
new features. Here is a short description of each patch file:


assert.patch

adds a switch to turn on/off the assert checking if enabled at compile
time. You can now compile postgres with assert checking and disable it
at runtime in a production environment.

async-unlisten.patch

declares Async_Unlisten() external so that it can be called by user
modules.

exec-limit.patch

removes the #ifdef NOT_USED around ExecutorLimit(). It is used.

exitpg.patch

limits recursive calls to exitpg() preventing an infinite loop
if an error is found inside exitpg.

libpgtcl-listen.patch

Just a change from upper to lowercase of an sql command in libpgtcl,
totally harmless.

new-locks.patch

After long studying and many debugging sessions I have finally
understood how the low level locks work.
I have completely rewritten lock.c cleaning up the code and adding
better assert checking. I have also added some fields to the lock
and xid tags for better support of user locks. This patch includes
also a patch submitted by Bruce Momjian which changes the handling
of lock priorities. It can however be disabled if an option is set
in pg_options, see tprintf.patch (Bruce patch works by building
the queue in reverse priority order, my old patch kept the queue in
decreasing order and traversed it from the other side).

pg-flush.patch

removes an unnecessary flush in libpq reducing network traffic and
increasing performance.

relname.patch

an utility which returns the relname corresponding to a given oid.
Useful for debug messages (see vacum.patch).

sequence.patch

added a setval() function which enables othe owner of a sequence
to set its value without need to delete and recreate it.

sinval.patch

fixes a problem in SI cache which causes table overflow if some
backend is idle for a long time while other backends keep adding
entries.
It uses the new signal handling implemented in tprintf.patch.
I have also increacasesed the max number of backends from 32 to 64 and
the table size from 1000 to 5000.

spin-lock.patch

I'm not sure if this is really useful, but it seems stupid to have
a backend wasting cpu cycles in a busy loop while the process which
should release the lock is waiting for the cpu. So I added a call
to process_yield() if the spin lock can't obtained.
This has been implemented and tested only on Linux. I don't know if
other OS have process_yield(). If someone can check please do it.

tprintf.patch

adds functions and macros which implement a conditional trace package
with the ability to change flags and numeric options of running
backends at runtime.
Options/flags can be specified in the command line and/or read from
the file pg_options in the data directory.
Running backends can be forced to update their options from this file
by sending them a SIGHUP signal (this is the convention used by most
unix daemons so I changed the meaning of SIGHUP).
Options can be debugging flags used by the trace package or any other
numeric value used by the backend, for example the deadlock_timeout.
Having flags and options specified at runtime and changed while the
backends are running can greatly simplify the debugging and tuning
of the database. New options can be defined in utils/misc/trace.c and
include/utils/trace.h. As an example of the usage of this package
see lock.c and proc.c which make use of new runtime options.

Old files using int flags or variables can be easily changed to
use the new package by substituting the old variable with a #define
like in the following example:

/* int my_flag = 0; */
#include "trace.h"
#define my_flag pg_options[OPT_MYFLAG]

I have done it in postgres.c and some other files and now I can turn
on/off any single debug flag on the fly with a simple shell script.
I have removed the IpcConfigTip() from ipc.c, it should better be
described in the postgres manual instead of being printed on stderr.

This patch provides also a new format of debugging messages which
are always in a single line with a timestamp and the backend pid:

#timestamp #pid #message
980127.17:52:14.173 [29271] StartTransactionCommand
980127.17:52:14.174 [29271] ProcessUtility: drop table t;
980127.17:52:14.186 [29271] SIIncNumEntries: table is 70% full
980127.17:52:14.186 [29286] Async_NotifyHandler
980127.17:52:14.186 [29286] Waking up sleeping backend process
980127.19:52:14.292 [29286] Async_NotifyFrontEnd
980127.19:52:14.413 [29286] Async_NotifyFrontEnd done
980127.19:52:14.466 [29286] Async_NotifyHandler done

This improves the readability of the log and allows one to understand
exactly which backend is doing what and at which time. It also makes
easier to write simple awk or perl scripts which monitor the log to
detect database errors or problem, or to compute transaction times.

The patch changes also the meaning of signals used by postgres, as
described by the following table:

postmaster backend

SIGHUP kill(*,sighup) read_pg_options
SIGINT kill(*,sigint), die die
SIGCHLD reaper -
SIGTTIN ignored -
SIGTTOU ignored -
SIGQUIT die handle_warn
SIGTERM kill(*,sigterm), kill(*,9), die die
SIGCONT dumpstatus -
SIGPIPE ignored die
SIGFPE - FloatExceptionHandler
SIGTSTP - ignored (alive test)
SIGUSR1 kill(*,sigusr1), die quickdie
SIGUSR2 kill(*,sigusr2) Async_NotifyHandler
(also SI buffer flush)

The main changes to the old implementation are SIGQUIT instead of
SIGHUP to handle warns, SIGHUP to reread pg_options and redirection
to all backends of SIGHUP, SIGINT, SIGTERM, SIGUSR1 and SIGUSR2.
In this way some of the signals sent to the postmaster can be sent
automatically to all the backends. To shut down postgres one needs
only to send a SIGTERM to postmaster and it will stop automatically
all the backends. This new signal handling mechanism is also used
to prevent SI cache table overflows: when a backend detects the SI
table full at 70% it simply sends a signal to the postmaster which
will wake up all idle backends and make them flush the cache.

vacuum.patch

adds a debug message to vacuum that prints the name of a table or
index *before* vacuuming it, if the verbose keyword is set.
This is useful to know which table is causing troubles if a
vacuum all crashes. Currently table information is printed only
at the end of each vacuum operation and is never printed if the
vacuum crashes.

--
Massimo Dal Zotto