FAQ
OK, we need to vote on this patch. It reduces the tuple header by 4
bytes (11% decrease).

If we apply it, we will not be able to easily use pg_upgrade for 7.3
because the on-disk table format will change.

Votes are:

1) Apply it now
2) Wait until August and see if any other table format changes are made.
3) Delay patch until we have other table format changes.


---------------------------------------------------------------------------

Manfred Koizar wrote:
On Fri, 14 Jun 2002 10:16:22 -0400, Tom Lane wrote:
As I commented before, I am not in favor of this. I don't think that a
four-byte savings justifies a forced initdb with no chance of
pg_upgrade,
As I don't know other users' preferences, I cannot comment on this.
I just think that four bytes per tuple can amount to a sum that justifies
this effort. Disk space is not my argument here, but reduced disk IO is.
plus significantly slower access to
several critical header fields. The tqual.c routines are already
hotspots in many scenarios. I believe this will make them noticeably
slower.
Significantly slower? I tried to analyze HeapTupleSatisfiesUpdate(),
as I think it is the most complicated of these Satisfies functions
(look for the !! comments):


So in the worst case we return after having done four more
compares than without the patch. Note that in the most common
cases there is no additional cost at all. If you still think
we have a performance problem here, we could replace GetCmin
and GetCmax by cheaper macros:

#define HeapTupleHeaderGetCminKnowingThatNotMoved(tup) \
( \
AssertMacro(!((tup)->t_infomask & HEAP_MOVED)),
( \
((tup)->t_infomask & (HEAP_XMIN_IS_XMAX | HEAP_XMAX_INVALID)) ? \
(CommandId) (tup)->t_cid \
: \
FirstCommandId \
) \
)

thus reducing the additional cost to one t_infomask compare,
because the Satisfies functions only access Cmin and Cmax,
when HEAP_MOVED is known to be not set.

OTOH experimenting with a moderatly sized "out of production"
database I got the following results:
pages | pages |
relkind | count | tuples | before| after | savings
--------+-------+--------+-------+-------+--------
i | 31 | 808146 | 8330 | 8330 | 0.00%
r | 32 | 612968 | 13572 | 13184 | 2.86%
all | 63 | | 21902 | 21514 | 1.77%

2.86% fewer heap pages mean 2.86% less disk IO caused by heap pages.
Considering that index pages tend to benefit more from caching
we conclude that heap pages contribute more to the overall
IO load, so the total savings in the number of disk IOs should
be better than the 1.77% shown in the table above. I think
this outweighs a few CPU cycles now and then.

Servus
Manfred

---------------------------(end of broadcast)---------------------------
TIP 2: you can get off all lists at once with the unregister command
(send "unregister YourEmailAddressHere" to majordomo@postgresql.org)
--
Bruce Momjian | http://candle.pha.pa.us
pgman@candle.pha.pa.us | (610) 853-3000
+ If your life is a hard drive, | 830 Blythe Avenue
+ Christ can be your backup. | Drexel Hill, Pennsylvania 19026

Search Discussions

  • Neil Conway at Jun 28, 2002 at 11:54 pm

    On Fri, Jun 28, 2002 at 07:32:00PM -0400, Bruce Momjian wrote:
    OK, we need to vote on this patch. It reduces the tuple header by 4
    bytes (11% decrease).

    If we apply it, we will not be able to easily use pg_upgrade for 7.3
    because the on-disk table format will change.
    I vote for apply it now -- if there are no other on-disk format changes
    made by late August and pg_upgrade is actually a valid, production-quality
    upgrade mechanism (which I'm not prepared to assume), consider
    reverting it.

    Cheers,

    Neil

    --
    Neil Conway <neilconway@rogers.com>
    PGP Key ID: DB3C29FC
  • Curt Sampson at Jul 1, 2002 at 2:50 am

    On Fri, 28 Jun 2002, Bruce Momjian wrote:

    OK, we need to vote on this patch. It reduces the tuple header by 4
    bytes (11% decrease).

    If we apply it, we will not be able to easily use pg_upgrade for 7.3
    because the on-disk table format will change.

    Votes are:

    1) Apply it now
    2) Wait until August and see if any other table format changes are made.
    3) Delay patch until we have other table format changes.
    I would tend to say "apply it now" so that we can get more testing
    of it.

    It would also be good to see how else we could save space in the
    header, e.g., by not having an empty OID field when a table is
    created without OIDs. (That would double the space savings.)

    I tend to use ID cross reference tables quite a lot, and these tend to
    have a lot of rows in them. (E.g., group table has group ID; user table
    has user-id; a group-id + user-id table determines which users are in
    which groups. In one project a couple of years ago, such a table was 85
    million rows.) These types of tables are typically 8 bytes of data and
    40 or so bytes of overhead. Ouch!

    cjs
    --
    Curt Sampson <cjs@cynic.net> +81 90 7737 2974 http://www.netbsd.org
    Don't you know, in this new Dark Age, we're all light. --XTC
  • Bruce Momjian at Jul 1, 2002 at 2:18 pm

    Curt Sampson wrote:
    On Fri, 28 Jun 2002, Bruce Momjian wrote:

    OK, we need to vote on this patch. It reduces the tuple header by 4
    bytes (11% decrease).

    If we apply it, we will not be able to easily use pg_upgrade for 7.3
    because the on-disk table format will change.

    Votes are:

    1) Apply it now
    2) Wait until August and see if any other table format changes are made.
    3) Delay patch until we have other table format changes.
    I would tend to say "apply it now" so that we can get more testing
    of it.
    OK, I have heard enough votes to add this. If more votes come in while
    it is in the queue, we can reevaluate.

    Also, Manfred is working on making the OID field optional, so it seems
    we may have more format changes coming. Time to focus on any other data
    format changes we want to be in 7.3.

    --
    Bruce Momjian | http://candle.pha.pa.us
    pgman@candle.pha.pa.us | (610) 853-3000
    + If your life is a hard drive, | 830 Blythe Avenue
    + Christ can be your backup. | Drexel Hill, Pennsylvania 19026
  • Manfred Koizar at Jul 1, 2002 at 3:37 pm

    On Mon, 1 Jul 2002 10:15:42 -0400 (EDT), Bruce Momjian wrote:
    OK, I have heard enough votes to add this.
    In a second version of this patch posted on 2002-06-26 you can control
    the tuple format by #define/#undef PG72FORMAT. While there have been
    voices saying that exposing this choice to the user via
    configure --enable-pg72format
    is not a good idea [well, it was one voice and the idea was called
    "really, really bad" ;-) but the argument is still valid], I wonder
    whether we shouldn't apply this second version (without the configure
    parts) and put all forthcoming format changes under #ifndef
    PG72FORMAT.

    This way you can decide to go back to v7.2 format immediately before
    going beta, if the changes are too hot to handle. And I think if I
    wouldn't volunteer to cleanup the #ifdef PG72FORMAT stuff after the
    new format has been accepted, I would be nominated to do it ...

    Servus
    Manfred
  • Tom Lane at Jul 1, 2002 at 5:10 pm

    Manfred Koizar writes:
    ... I wonder
    whether we shouldn't apply this second version (without the configure
    parts) and put all forthcoming format changes under #ifndef
    PG72FORMAT.
    Seems reasonable. I generally dislike #ifdef clutter, but the #ifs
    would only be around a couple of macro definitions AFAICT, so the
    readability hit would be minimal. And someone who wanted
    back-compatibility would be able to have it, whichever way we jump
    on the decision for 7.3.

    At the rate Manfred is going, he'll have patches for all the tuple and
    page header related issues before August anyway ... so my original gripe
    about wanting to group those changes into a single release will become
    moot ;-). I certainly have no objection to doing them all in 7.3
    instead of 7.4 if we can get it done.

    regards, tom lane
  • Bruce Momjian at Jul 2, 2002 at 6:19 am

    Tom Lane wrote:
    Manfred Koizar <mkoi-pg@aon.at> writes:
    ... I wonder
    whether we shouldn't apply this second version (without the configure
    parts) and put all forthcoming format changes under #ifndef
    PG72FORMAT.
    Seems reasonable. I generally dislike #ifdef clutter, but the #ifs
    would only be around a couple of macro definitions AFAICT, so the
    readability hit would be minimal. And someone who wanted
    back-compatibility would be able to have it, whichever way we jump
    on the decision for 7.3.
    I committed the version with no #ifdef's. If we need them, we can add
    them later, but it is likely we will never need them.
    At the rate Manfred is going, he'll have patches for all the tuple and
    page header related issues before August anyway ... so my original gripe
    about wanting to group those changes into a single release will become
    moot ;-). I certainly have no objection to doing them all in 7.3
    instead of 7.4 if we can get it done.
    Yes. Manfred, keep going. ;-)

    --
    Bruce Momjian | http://candle.pha.pa.us
    pgman@candle.pha.pa.us | (610) 853-3000
    + If your life is a hard drive, | 830 Blythe Avenue
    + Christ can be your backup. | Drexel Hill, Pennsylvania 19026
  • Manfred Koizar at Jul 3, 2002 at 10:27 am

    On Tue, 2 Jul 2002 02:16:29 -0400 (EDT), Bruce Momjian wrote:
    I committed the version with no #ifdef's. If we need them, we can add
    them later, but it is likely we will never need them.
    My point was, if there is a need to fallback to v7.2 format, it can be
    done by changing a single line from #undef to #define. IMO the next
    patch I'm going to submit is a bit more risky. But if everyone else
    is confident we can make it stable for v7.3, it's fine by me too.
    Yes. Manfred, keep going. ;-)
    Can't guarantee to keep the rate. You know, the kids need a bit more
    attention when they don't go to school :-)

    Servus
    Manfred
  • Bruce Momjian at Jul 3, 2002 at 5:22 pm

    Manfred Koizar wrote:
    On Tue, 2 Jul 2002 02:16:29 -0400 (EDT), Bruce Momjian
    wrote:
    I committed the version with no #ifdef's. If we need them, we can add
    them later, but it is likely we will never need them.
    My point was, if there is a need to fallback to v7.2 format, it can be
    done by changing a single line from #undef to #define. IMO the next
    patch I'm going to submit is a bit more risky. But if everyone else
    is confident we can make it stable for v7.3, it's fine by me too.
    Yes, with your recent pages, I think we are committed to changing the
    format for 7.3.
    Yes. Manfred, keep going. ;-)
    Can't guarantee to keep the rate. You know, the kids need a bit more
    attention when they don't go to school :-)
    Let me send over my kids. Where are you located? Austria? Hmmm...

    --
    Bruce Momjian | http://candle.pha.pa.us
    pgman@candle.pha.pa.us | (610) 853-3000
    + If your life is a hard drive, | 830 Blythe Avenue
    + Christ can be your backup. | Drexel Hill, Pennsylvania 19026

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouppgsql-hackers @
categoriespostgresql
postedJun 28, '02 at 11:32p
activeJul 3, '02 at 5:22p
posts9
users5
websitepostgresql.org...
irc#postgresql

People

Translate

site design / logo © 2021 Grokbase