FAQ
Well, I hope that bug reported by Michael is old bug.

Simple example:

bt=> create table test (x text);
CREATE
bt=> create index test_i on test (x);
CREATE
bt=> insert into test values ('0');
INSERT 449464
bt=> copy test from '/home/postgres/My/Btree/BUG/B';
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
400 records with '12345'
COPY
bt=> vacuum verbose test;
NOTICE:Rel test: Pages 4: Changed 4, Reapped 0, Empty 0, New 0; Tup 401: Vac 0, Crash 0, UnUsed 0, MinLen 53, MaxLen 57; Re-using: Free/Avail. Space 0/0; EndEmpty/Avail. Pages 0/0. Elapsed 0/0 sec.
NOTICE:Ind test_i: Pages 5; Tuples 400. Elapsed 0/0 sec.
NOTICE:Ind test_i: NUMBER OF INDEX' TUPLES (400) IS NOT THE SAME AS HEAP' (401)
VACUUM

(I've got the same results under 6.0)

: splitting occures when there are 1 '0' and 291 '12345' on the first btree
page - PageGetFreeSpace(page) returns 0. Following duplicates handling
rules, '0' has to go on the 1st page and 291 '12345' - on the new page.
+ new '12345' item will also go on the new page!!!
But there is no space for '12345' here: we got rid of '0' - 8 bytes of
data - and want to add 16 bytes (double aligning).
New item wasn't added, silently. Check added now, so you'll
get FATAL...

There is no time to fix it now. I'll try to do it latter.

...Original Lehman and Yao algorithm doesn't allow duplicates at all.
Old team added some hacks... and we have to add yet more...

BTW, there shouldn't be problems in using btree for variable
length attributes if there are no duplicates... Michael, are there
duplicates in your table ?

Marc, did you use btree for variable length attributes?
Do you have duplicates ?

Vadim

------------------------------

Search Discussions

  • Michael Reifenberger at Jun 6, 1997 at 8:46 am

    On Fri, 6 Jun 1997, Vadim B. Mikheev wrote:

    Date: Fri, 06 Jun 1997 15:19:52 +0800
    From: "Vadim B. Mikheev" <vadim@sable.krasnoyarsk.su>
    To: Michael Reifenberger <root@totum.plaut.de>
    Cc: PostgreSQL Hackers <hackers@postgreSQL.org>, scrappy@hub.org
    Subject: Re: [HACKERS] FATAL 1:btree: items are out of order

    Well, I hope that bug reported by Michael is old bug. ...
    BTW, there shouldn't be problems in using btree for variable
    length attributes if there are no duplicates... Michael, are there
    duplicates in your table ?
    Yes.

    - -- Lets look for duplicates.
    - -- And we have lots of them
    SELECT count(*) FROM h;
    ...
    (341224 rows)

    SELECT DISTINCT firmennummer FROM h;
    ...
    (111060 rows)

    SELECT DISTINCT imp_exp_code FROM h;
    ...
    (5 rows)

    SELECT DISTINCT position FROM h;
    ...
    (79 rows)

    SELECT DISTINCT branchen FROM h;
    ...
    (15200 rows)

    - ---------------------------

    If there is a bug regarding dublicates and btrees, why does work a:
    CREATE TABLE ...
    COPY ... FROM ...
    CREATE INDEX ...

    Does this bug occure only during inserts.

    If the bug can't be fixed for 6.1 then he should be documented.
    (Any chance to reproduce it in a regression test?)


    Thanks anyway for responding.


    Bye!
    - ----
    Michael Reifenberger
    Plaut Software GmbH, R/3 Basis

    PS: Do you still need my debugging-info you requested before?

    ------------------------------
  • Michael Reifenberger at Jun 6, 1997 at 11:30 am

    On Fri, 6 Jun 1997, Vadim B. Mikheev wrote:

    Date: Fri, 06 Jun 1997 19:31:48 +0800
    From: "Vadim B. Mikheev" <vadim@sable.krasnoyarsk.su>
    To: Michael Reifenberger <root@totum.plaut.de>
    Cc: PostgreSQL Hackers <hackers@postgreSQL.org>, scrappy@hub.org
    Subject: Re: [HACKERS] FATAL 1:btree: items are out of order ...
    No, but please try run new posted code (with checks after PageAddItem) -
    I'm interested in FATAL message in your case.
    Are these changes in 970506 ?

    Is there a way for getting the CVS-Tree (best via cvsup)?

    Bye!
    - ----
    Michael Reifenberger
    Plaut Software GmbH, R/3 Basis

    ------------------------------
  • Vadim B. Mikheev at Jun 6, 1997 at 11:31 am

    Michael Reifenberger wrote:

    On Fri, 6 Jun 1997, Vadim B. Mikheev wrote:
    ...
    BTW, there shouldn't be problems in using btree for variable
    length attributes if there are no duplicates... Michael, are there
    duplicates in your table ?
    Yes.

    If there is a bug regarding dublicates and btrees, why does work a:
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    btree + variable-length attributes + duplicates
    CREATE TABLE ...
    COPY ... FROM ...
    CREATE INDEX ...

    Does this bug occure only during inserts.
    Yes (I hope) - bulkload code don't split pages.
    If the bug can't be fixed for 6.1 then he should be documented.
    (Any chance to reproduce it in a regression test?)

    PS: Do you still need my debugging-info you requested before?
    No, but please try run new posted code (with checks after PageAddItem) -
    I'm interested in FATAL message in your case.

    Vadim

    ------------------------------
  • Michael Reifenberger at Jun 6, 1997 at 1:10 pm

    On Fri, 6 Jun 1997, Vadim B. Mikheev wrote:

    Date: Fri, 06 Jun 1997 19:31:48 +0800
    From: "Vadim B. Mikheev" <vadim@sable.krasnoyarsk.su>
    To: Michael Reifenberger <root@totum.plaut.de>
    Cc: PostgreSQL Hackers <hackers@postgreSQL.org>, scrappy@hub.org
    Subject: Re: [HACKERS] FATAL 1:btree: items are out of order

    Michael Reifenberger wrote: ...
    No, but please try run new posted code (with checks after PageAddItem) -
    I'm interested in FATAL message in your case.
    FATAL 1:btree: items are out of order (leftmost 0, stack 29, update 2)

    Bye!
    - ----
    Michael Reifenberger
    Plaut Software GmbH, R/3 Basis

    ------------------------------
  • The Hermit Hacker at Jun 6, 1997 at 9:20 pm

    On Fri, 6 Jun 1997, Vadim B. Mikheev wrote:

    Well, I hope that bug reported by Michael is old bug.

    Simple example:

    bt=> create table test (x text);
    CREATE
    bt=> create index test_i on test (x);
    CREATE
    bt=> insert into test values ('0');
    INSERT 449464
    bt=> copy test from '/home/postgres/My/Btree/BUG/B';
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    400 records with '12345'
    COPY
    bt=> vacuum verbose test;
    NOTICE:Rel test: Pages 4: Changed 4, Reapped 0, Empty 0, New 0; Tup 401: Vac 0, Crash 0, UnUsed 0, MinLen 53, MaxLen 57; Re-using: Free/Avail. Space 0/0; EndEmpty/Avail. Pages 0/0. Elapsed 0/0 sec.
    NOTICE:Ind test_i: Pages 5; Tuples 400. Elapsed 0/0 sec.
    NOTICE:Ind test_i: NUMBER OF INDEX' TUPLES (400) IS NOT THE SAME AS HEAP' (401)
    VACUUM

    (I've got the same results under 6.0)

    : splitting occures when there are 1 '0' and 291 '12345' on the first btree
    page - PageGetFreeSpace(page) returns 0. Following duplicates handling
    rules, '0' has to go on the 1st page and 291 '12345' - on the new page.
    + new '12345' item will also go on the new page!!!
    But there is no space for '12345' here: we got rid of '0' - 8 bytes of
    data - and want to add 16 bytes (double aligning).
    New item wasn't added, silently. Check added now, so you'll
    get FATAL...

    There is no time to fix it now. I'll try to do it latter.

    ...Original Lehman and Yao algorithm doesn't allow duplicates at all.
    Old team added some hacks... and we have to add yet more...

    BTW, there shouldn't be problems in using btree for variable
    length attributes if there are no duplicates... Michael, are there
    duplicates in your table ?

    Marc, did you use btree for variable length attributes?
    Do you have duplicates ?
    Assuming you mean varchar()? No, all my attributes are of text...
    but, then again, I guess that means variable lenght, eh? :( Yes, my
    indices are based on variable length attributes...

    As for duplicates...the most likely are duplicates for each
    'indexed field'..


    Marc G. Fournier
    Systems Administrator @ hub.org
    primary: scrappy@hub.org secondary: scrappy@{freebsd|postgresql}.org

    ------------------------------
  • Bruce Momjian at Jun 6, 1997 at 9:26 pm

    There is no time to fix it now. I'll try to do it latter.

    ...Original Lehman and Yao algorithm doesn't allow duplicates at all.
    Old team added some hacks... and we have to add yet more...

    BTW, there shouldn't be problems in using btree for variable
    length attributes if there are no duplicates... Michael, are there
    duplicates in your table ?

    Marc, did you use btree for variable length attributes?
    Do you have duplicates ?
    Assuming you mean varchar()? No, all my attributes are of text...
    but, then again, I guess that means variable lenght, eh? :( Yes, my
    indices are based on variable length attributes...

    As for duplicates...the most likely are duplicates for each
    'indexed field'..
    The only thing I want to know is, "Do we need to add this to the TODO
    bugs section, so people know this doesn't work, or is this a rare thing?"

    - --
    Bruce Momjian
    maillist@candle.pha.pa.us

    ------------------------------
  • Vadim B. Mikheev at Jun 7, 1997 at 6:15 am
    I found from where problem comes. It concerns duplicates,
    not variable-length attribute (var(XXX) is fixed-length attr, Marc).

    Reproduction under 6.1-current:

    drop table bug_duplicate;
    create table bug_duplicate (x int);
    create index bug_duplicate_i on bug_duplicate (x);
    copy bug_duplicate from '/home/postgres/My/Btree/BUG/I2';
    - -- ^^
    - -- 408 (!) records with x = 2
    vacuum bug_duplicate;

    select count(*) from bug_duplicate where x = 2;
    count
    - -----
    408
    (1 row)

    insert into bug_duplicate values (1);
    INSERT 1867199
    select count(*) from bug_duplicate where x = 2;
    count
    - -----
    203
    (1 row)

    =============

    Under 6.0:

    drop table bug_duplicate;
    create table bug_duplicate (x int);
    create index bug_duplicate_i on bug_duplicate (x);
    copy bug_duplicate from '/tmp/I2';
    - -- ^^
    - -- 292 (!) records with x = 2
    vacuum bug_duplicate;
    select count(*) from bug_duplicate where x = 2;
    count
    - -----

    292
    (1
    row)

    insert into bug_duplicate values (1);
    select count(*) from bug_duplicate where x = 2;
    count
    - -----

    146
    (1
    row)

    ===============

    This examples are not the same as

    drop table bug_duplicate;
    create table bug_duplicate (x int);
    create index bug_duplicate_i on bug_duplicate (x);
    copy bug_duplicate from '/home/postgres/My/Btree/BUG/I2';
    - -- ^^
    - -- = 450 records with x = 2
    vacuum bug_duplicate;

    And now:
    vac=> select count(*) from bug_duplicate where x = 2;
    count
    - -----
    450
    (1 row)

    vac=> insert into bug_duplicate values (1);
    INSERT 1695826
    vac=> select count(*) from bug_duplicate where x = 2;
    count
    - -----
    204
    (1 row)

    : it's fixed currently - having 450 items with 2 splitting occures
    before we inserts item with 1, - but in the new example
    splitting occures when we insert item with 1.

    =============

    Ok. Found but unfixed, yet. I suggest to postpone release for
    2-3 day and got to sleep.

    Vadim

    ------------------------------
  • The Hermit Hacker at Jun 7, 1997 at 3:19 pm

    On Sat, 7 Jun 1997, Vadim B. Mikheev wrote:

    Ok. Found but unfixed, yet. I suggest to postpone release for
    2-3 day and got to sleep.
    so, sunday instead of friday? :)

    I'll leave this one as 'the last bug to fix'...as soon as you can get
    it fixed, we'll do up the release, since, if nothing else, it will prevent
    ppl from asking about the NOTICE :)


    Marc G. Fournier
    Systems Administrator @ hub.org
    primary: scrappy@hub.org secondary: scrappy@{freebsd|postgresql}.org

    ------------------------------
  • Vadim B. Mikheev at Jun 10, 1997 at 8:11 am
    Hi!

    New btree code in CVS!
    Sorry for lateness - it took 28 hours of continuous work +
    3 hours of final tests.
    All my tests are OK (I even played with 160000 of duplicates).
    Also, I hope that variable-length attributes support improved -
    at least the example below now works:
    bt=> create table test (x text);
    CREATE
    bt=> create index test_i on test (x);
    CREATE
    bt=> insert into test values ('0');
    INSERT 449464
    bt=> copy test from '/home/postgres/My/Btree/BUG/B';
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    400 records with '12345'
    COPY
    bt=> vacuum verbose test;
    NOTICE:Rel test: Pages 4: Changed 4, Reapped 0, Empty 0, New 0; Tup 401: Vac 0, Crash 0, UnUsed 0, MinLen 53, MaxLen 57; Re-using: Free/Avail. Space 0/0; EndEmpty/Avail. Pages 0/0. Elapsed 0/0 sec.
    NOTICE:Ind test_i: Pages 5; Tuples 400. Elapsed 0/0 sec.
    NOTICE:Ind test_i: NUMBER OF INDEX' TUPLES (400) IS NOT THE SAME AS HEAP' (401)
    VACUUM
    I posted new code to Michael and hope that he will test them soon
    in real-life example.

    Note, that for safety dump/initdb/reload required - previous fix
    has bugs!

    Vadim

    ------------------------------
  • Michael Reifenberger at Jun 10, 1997 at 11:45 am
    Hi,

    On Tue, 10 Jun 1997, Vadim B. Mikheev wrote:
    ...
    New btree code in CVS!
    Sorry for lateness - it took 28 hours of continuous work +
    3 hours of final tests.
    All my tests are OK (I even played with 160000 of duplicates).
    Also, I hope that variable-length attributes support improved -
    at least the example below now works:
    First of all, thanks all the effords.
    The new patches are going into the right direction and work partially.

    I created all bins and DB's from scratch.

    If I create the table h and an index h1,
    where h1 consists of ONE variable field of type text_ops,
    then it works to copy the table!

    Thats the good news.

    If the index consists of MORE (3 here) variable text fields,
    than I get:

    hopp=> copy h from '/a/hopp/export/h.dmp';
    FATAL 1:btree: items are out of order (leftmost 0, stack 29, update 2)



    Thanks so far.


    Bye!
    - ----
    Michael Reifenberger
    Plaut Software GmbH, R/3 Basis

    ------------------------------
  • Bruce Momjian at Jun 10, 1997 at 12:53 pm

    Hi!

    New btree code in CVS!
    Sorry for lateness - it took 28 hours of continuous work +
    3 hours of final tests.
    All my tests are OK (I even played with 160000 of duplicates).
    Also, I hope that variable-length attributes support improved -
    at least the example below now works:
    Wow, 28 hours. That's a lot of time. Thanks.

    - --
    Bruce Momjian
    maillist@candle.pha.pa.us

    ------------------------------
  • Marc G. Fournier at Jun 10, 1997 at 7:45 pm

    On Tue, 10 Jun 1997, Bruce Momjian wrote:


    Hi!

    New btree code in CVS!
    Sorry for lateness - it took 28 hours of continuous work +
    3 hours of final tests.
    All my tests are OK (I even played with 160000 of duplicates).
    Also, I hope that variable-length attributes support improved -
    at least the example below now works:
    Wow, 28 hours. That's a lot of time. Thanks.
    Am I allowed to put out a release tonight then? :) Please? :)

    Marc G. Fournier scrappy@hub.org
    Systems Administrator @ hub.org scrappy@freebsd.org

    ------------------------------
  • Vadim B. Mikheev at Jun 11, 1997 at 2:59 am
    Michael, do you use chars which >= 128 ?
    If yes and you havn't USE_LOCALE defined that problems
    with text-field come from nbtcompare.c:bttextcmp() - it uses

    char *ap, *bp;
    ^^^^
    (it was changed from unsigned near 20 May)

    while text funcs in varlena.c use unsigned char: UNSIGNED_CHAR_TEXT
    is defined. (It's big inconvenience that btree in one cases use
    funcs from nbtcompare.c and in other cases - from utils/adt/*.c)
    Do you remember bug with btree and AbsoluteTime - the same cause!

    I hope that my assumption is right and we may have good dreams
    with cleaned btree...

    But what should we use - signed or unsigned chars in both places
    if USE_LOCALE is not defined ?

    We must clean this code before 6.1! It's easy.

    Vadim

    ------------------------------

    End of hackers-digest V1 #382
    *****************************
  • Vadim B. Mikheev at Jun 11, 1997 at 5:33 am

    Vadim B. Mikheev wrote:

    Michael, do you use chars which >= 128 ?
    If yes and you havn't USE_LOCALE defined that problems
    with text-field come from nbtcompare.c:bttextcmp() - it uses

    char *ap, *bp;
    ^^^^
    (it was changed from unsigned near 20 May)

    while text funcs in varlena.c use unsigned char: UNSIGNED_CHAR_TEXT
    is defined. (It's big inconvenience that btree in one cases use
    Reproduction:

    File X generated from
    #!/usr/local/bin/perl

    for ($i = 1; $i <= 1000; $i++)
    {
    $val = rand (255);
    next if $val < 32;
    printf "%c\n", $val;
    }

    now:
    vac=> create table t (x text);
    CREATE
    vac=> create index ti on t (x);
    CREATE
    vac=> copy t from '/home/postgres/My/OPTIMIZER/X';
    FATAL 1:btree: items are out of order (leftmost 0, stack 3, update 1)

    For 'char' field:
    vac=> drop table t;
    DROP
    vac=> create table t (x char);
    CREATE
    vac=> create index ti on t (x);
    CREATE
    vac=> copy t from '/home/postgres/My/OPTIMIZER/X';
    FATAL 1:btree: items are out of order (leftmost 0, stack 3, update 1)

    If you create index AFTER copy (bulkload code):
    vac=> create table t (x char);
    CREATE
    vac=> copy t from '/home/postgres/My/OPTIMIZER/X';
    COPY
    vac=> create index ti on t (x);
    CREATE

    : bulkload code use functions from utils/adt/*.c and so - all ok, but:

    vac=> select count(*) from t where x = '9';
    count
    - -----

    (1 row)

    vac=> drop index ti;
    DROP
    vac=> select count(*) from t where x = '9';
    count
    - -----
    5
    (1 row)

    I'm going to change all to 'unsigned' in nbtcompare.c
    both for text and char types:

    1. UNSIGNED_CHAR_TEXT is defined by default in config.h
    for quite long time.
    2. strncmp is used for char2, char8, ...

    Vadim

    ------------------------------
  • Michael Reifenberger at Jun 11, 1997 at 7:09 am

    On Wed, 11 Jun 1997, Vadim B. Mikheev wrote:

    Michael, do you use chars which >= 128 ?
    If yes and you havn't USE_LOCALE defined that problems
    with text-field come from nbtcompare.c:bttextcmp() - it uses
    For the USE_LOCALE, I'm using the default setting that comes with 6.1.
    I'm not shure if it's on or off.

    For the >128, I'm also not shure, but maybe.
    Do you have a fast way for me to proof this?



    Bye!
    - ----
    Michael Reifenberger
    Plaut Software GmbH, R/3 Basis

    ------------------------------
  • Michael Reifenberger at Jun 11, 1997 at 7:33 am

    On Wed, 11 Jun 1997, Vadim B. Mikheev wrote:

    For the >128, I'm also not shure, but maybe.
    Do you have a fast way for me to proof this?
    =20
    Something like
    =20
    select * from _table_ where _text_field_ ~ '[=C1-=D1=E1-=F1]'
    ^ ^^ ^
    I used first/last small/big russian letters.
    AH! You mean char-values greater 127!=20
    (Sorry, it's still too early for me, and my mind is splitted
    between postgres and a corrupted DEC-Appha :-( )

    I'm shurely have some of them.
    My data came from a source where some fields seemed to be corrupted.

    =20
    Please try new nbtcompare.c, which follows (sorry).
    =20
    I'll do it soon/now.

    Wait a bit.

    Bye!
    - ----
    Michael Reifenberger
    Plaut Software GmbH, R/3 Basis

    ------------------------------
  • Michael Reifenberger at Jun 11, 1997 at 7:37 am
    On Wed, 11 Jun 1997, Vadim B. Mikheev wrote:
    ...
    I used first/last small/big russian letters.

    Please try new nbtcompare.c, which follows (sorry).
    Do you want me to USE_LOCALE for the test?

    Bye!
    - ----
    Michael Reifenberger
    Plaut Software GmbH, R/3 Basis

    ------------------------------
  • Vadim B. Mikheev at Jun 11, 1997 at 7:58 am
    This is a multi-part message in MIME format.

    - --------------2DE067D346CA4DC22CDB6A0D
    Content-Type: text/plain; charset=iso-8859-1
    Content-Transfer-Encoding: 8bit

    Michael Reifenberger wrote:
    On Wed, 11 Jun 1997, Vadim B. Mikheev wrote:

    Michael, do you use chars which >= 128 ?
    If yes and you havn't USE_LOCALE defined that problems
    with text-field come from nbtcompare.c:bttextcmp() - it uses
    For the USE_LOCALE, I'm using the default setting that comes with 6.1.
    I'm not shure if it's on or off.
    I believe that it's off by default.
    For the >128, I'm also not shure, but maybe.
    Do you have a fast way for me to proof this?
    Something like

    select * from _table_ where _text_field_ ~ '[Á-Ñá-ñ]'
    ^ ^^ ^
    I used first/last small/big russian letters.

    Please try new nbtcompare.c, which follows (sorry).

    Vadim

    - --------------2DE067D346CA4DC22CDB6A0D
    Content-Type: text/plain; charset=us-ascii; name="nbtcompare.c"
    Content-Transfer-Encoding: 7bit
    Content-Disposition: inline; filename="nbtcompare.c"

    /*-------------------------------------------------------------------------
    *
    * nbtcompare.c--
    * Comparison functions for btree access method.
    *
    * Copyright (c) 1994, Regents of the University of California
    *
    *
    * IDENTIFICATION
    * $Header: /usr/local/cvsroot/postgres95/src/backend/access/nbtree/nbtcompare.c,v 1.9 1997/05/22 00:07:15 scrappy Exp $
    *
    * NOTES
    * These functions are stored in pg_amproc. For each operator class
    * defined on btrees, they compute
    *
    * compare(a, b):
    * < 0 if a < b,
    * = 0 if a == b,
    * > 0 if a > b.
    *-------------------------------------------------------------------------
    */

    #include <string.h>

    #include <postgres.h>

    #include <utils/builtins.h>
    #include <utils/nabstime.h>

    int32
    btint2cmp(int16 a, int16 b)
    {
    return ((int32) (a - b));
    }

    int32
    btint4cmp(int32 a, int32 b)
    {
    return (a - b);
    }

    int32
    btint24cmp(int16 a, int32 b)
    {
    return (((int32) a) - b);
    }

    int32
    btint42cmp(int32 a, int16 b)
    {
    return (a - ((int32) b));
    }

    int32
    btfloat4cmp(float32 a, float32 b)
    {
    if (*a > *b)
    return (1);
    else if (*a == *b)
    return (0);
    else
    return (-1);
    }

    int32
    btfloat8cmp(float64 a, float64 b)
    {
    if (*a > *b)
    return (1);
    else if (*a == *b)
    return (0);
    else
    return (-1);
    }

    int32
    btoidcmp(Oid a, Oid b)
    {
    if (a > b)
    return (1);
    else if (a == b)
    return (0);
    else
    return (-1);
    }

    int32
    btabstimecmp(AbsoluteTime a, AbsoluteTime b)
    {
    if (AbsoluteTimeIsBefore(a, b))
    return (-1);
    else if (AbsoluteTimeIsBefore(b, a))
    return (1);
    else
    return (0);
    }

    int32
    btcharcmp(char a, char b)
    {
    return ((int32) ((uint8)a - (uint8)b));
    }

    int32
    btchar2cmp(uint16 a, uint16 b)
    {
    return (strncmp((char *) &a, (char *) &b, 2));
    }

    int32
    btchar4cmp(uint32 a, uint32 b)
    {
    return (strncmp((char *) &a, (char *) &b, 4));
    }

    int32
    btchar8cmp(char *a, char *b)
    {
    return (strncmp(a, b, 8));
    }

    int32
    btchar16cmp(char *a, char *b)
    {
    return (strncmp(a, b, 16));
    }

    int32
    btnamecmp(NameData *a, NameData *b)
    {
    return (strncmp(a->data, b->data, NAMEDATALEN));
    }

    int32
    bttextcmp(struct varlena *a, struct varlena *b)
    {
    int res;
    unsigned char *ap, *bp;

    #ifdef USE_LOCALE
    int la = VARSIZE(a) - VARHDRSZ;
    int lb = VARSIZE(b) - VARHDRSZ;

    ap = (unsigned char *) palloc (la + 1);
    bp = (unsigned char *) palloc (lb + 1);

    memcpy(ap, VARDATA(a), la);
    *(ap + la) = '\0';
    memcpy(bp, VARDATA(b), lb);
    *(bp + lb) = '\0';

    res = strcoll (ap, bp);

    pfree (ap);
    pfree (bp);

    #else
    int len = VARSIZE(a);

    /* len is the length of the shorter of the two strings */
    if ( len > VARSIZE(b) )
    len = VARSIZE(b);

    len -= VARHDRSZ;

    ap = (unsigned char *) VARDATA(a);
    bp = (unsigned char *) VARDATA(b);

    /*
    * If the two strings differ in the first len bytes, or if they're
    * the same in the first len bytes and they're both len bytes long,
    * we're done.
    */

    res = 0;
    if (len > 0) {
    do {
    res = (int) (*ap++ - *bp++);
    len--;
    } while (res == 0 && len != 0);
    }

    #endif

    if (res != 0 || VARSIZE(a) == VARSIZE(b))
    return (res);

    /*
    * The two strings are the same in the first len bytes, and they
    * are of different lengths.
    */

    if (VARSIZE(a) < VARSIZE(b))
    return (-1);
    else
    return (1);
    }

    - --------------2DE067D346CA4DC22CDB6A0D--

    ------------------------------
  • Vadim B. Mikheev at Jun 11, 1997 at 8:59 am

    Michael Reifenberger wrote:

    On Wed, 11 Jun 1997, Vadim B. Mikheev wrote:
    ...
    I used first/last small/big russian letters.

    Please try new nbtcompare.c, which follows (sorry).
    Do you want me to USE_LOCALE for the test?
    No! It's fixed for non-USE_LOCALE. Just re-compile with
    new nbtcompare.c...
    (With USE_LOCALE all should be Ok too.)

    Vadim

    ------------------------------
  • Michael Reifenberger at Jun 11, 1997 at 9:57 am
    On Wed, 11 Jun 1997, Vadim B. Mikheev wrote:
    ...
    Please try new nbtcompare.c, which follows (sorry).
    Do you want me to USE_LOCALE for the test?
    No! It's fixed for non-USE_LOCALE. Just re-compile with
    new nbtcompare.c...
    (With USE_LOCALE all should be Ok too.)
    Ok, I tried.

    One time it worked,
    three times it failed.

    Same definition, same data, same programs.

    hopp=> copy h from '/a/hopp/export/h.dmp';
    FATAL 1:btree: items are out of order (leftmost 0, stack 59, update 2)

    hopp=> copy h from '/a/hopp/export/h.dmp';
    FATAL 1:btree: items are out of order (leftmost 0, stack 68, update 2)

    hopp=> copy h from '/a/hopp/export/h.dmp~';
    FATAL 1:btree: items are out of order (leftmost 0, stack 29, update 2)

    If I look onto the datafile-size then it seems to break on different places.

    Sorry,


    Bye!
    - ----
    Michael Reifenberger
    Plaut Software GmbH, R/3 Basis

    ------------------------------
  • Michael Reifenberger at Jun 11, 1997 at 1:25 pm
    On Wed, 11 Jun 1997, Vadim B. Mikheev wrote:
    ...
    Michael, do you use chars which >= 128 ?
    If yes and you havn't USE_LOCALE defined that problems
    with text-field come from nbtcompare.c:bttextcmp() - it uses
    Next try.
    Now I compiled the source with (configuere --enable-locale).

    createdb/create_table/create_index

    Then:
    hopp=> copy h from '/a/hopp/export/h.dmp';
    FATAL 1:palloc failure: memory exhausted

    (But I have 64MB Memory + 588MB Swap, which is 11% used).

    After restarting psql again I try:

    hopp=> drop table h;
    NOTICE:AbortTransaction and not in in-progress state
    NOTICE:AbortTransaction and not in in-progress state

    ????

    Not only that.

    It seems that the files h (the table) and h1 (the index) get deleted.
    (As it should)
    If I now recreate the table and index, I get a message that they already exist.
    (But the datafiles don't).

    If I do a \d, I see woth index and table, if I do a select * from h I get;
    WARN:cannot write block 1 of h1 [hopp] blind

    (Which is correct because the datafiles are missing).


    Bye!
    - ----
    Michael Reifenberger
    Plaut Software GmbH, R/3 Basis

    ------------------------------
  • Darren King at Jun 11, 1997 at 1:33 pm

    Michael, do you use chars which >= 128 ?
    If yes and you havn't USE_LOCALE defined that problems
    with text-field come from nbtcompare.c:bttextcmp() - it uses

    char *ap, *bp;
    ^^^^
    (it was changed from unsigned near 20 May)

    while text funcs in varlena.c use unsigned char: UNSIGNED_CHAR_TEXT
    is defined. (It's big inconvenience that btree in one cases use
    funcs from nbtcompare.c and in other cases - from utils/adt/*.c)
    Do you remember bug with btree and AbsoluteTime - the same cause!

    I hope that my assumption is right and we may have good dreams
    with cleaned btree...

    But what should we use - signed or unsigned chars in both places
    if USE_LOCALE is not defined ?

    We must clean this code before 6.1! It's easy.

    Would using unsigned chars for varlena break anything else? If not,
    I think we should use unsigned since it is necessary for USE_LOCALE
    and wouldn't hurt if not USE_LOCALE.

    Darren darrenk@insightdist.com

    ------------------------------
  • Vadim B. Mikheev at Jun 12, 1997 at 12:48 am

    Darren King wrote:
    Michael, do you use chars which >= 128 ?
    If yes and you havn't USE_LOCALE defined that problems
    with text-field come from nbtcompare.c:bttextcmp() - it uses

    char *ap, *bp;
    ^^^^
    (it was changed from unsigned near 20 May)

    while text funcs in varlena.c use unsigned char: UNSIGNED_CHAR_TEXT
    is defined. (It's big inconvenience that btree in one cases use
    funcs from nbtcompare.c and in other cases - from utils/adt/*.c)
    Do you remember bug with btree and AbsoluteTime - the same cause!

    I hope that my assumption is right and we may have good dreams
    with cleaned btree...

    But what should we use - signed or unsigned chars in both places
    if USE_LOCALE is not defined ?

    We must clean this code before 6.1! It's easy.
    Would using unsigned chars for varlena break anything else? If not,
    I think we should use unsigned since it is necessary for USE_LOCALE
    and wouldn't hurt if not USE_LOCALE.
    I didn't change varlena but I'd change nbtcompare.c already.

    Vadim

    ------------------------------

    End of hackers-digest V1 #383
    *****************************
  • Vadim B. Mikheev at Jun 12, 1997 at 12:55 pm
    Hi!

    Sorry, I left office 11 Jun before Michael got bad results
    with new nbtcompare.c and I forgot to say that 12 Jun is holyday
    in Russia. Now I'm here and waiting for test data from Michael...
    BTW, my test with char-values > 127 is Ok.

    Michael Reifenberger wrote:
    On Wed, 11 Jun 1997, Vadim B. Mikheev wrote:
    ...
    Michael, do you use chars which >= 128 ?
    If yes and you havn't USE_LOCALE defined that problems
    with text-field come from nbtcompare.c:bttextcmp() - it uses
    Next try.
    Now I compiled the source with (configuere --enable-locale).

    createdb/create_table/create_index

    Then:
    hopp=> copy h from '/a/hopp/export/h.dmp';
    FATAL 1:palloc failure: memory exhausted
    I don't understand it! bttextcmp() frees memory allocated
    for strcoll! Problems in strcoll ?
    (But I have 64MB Memory + 588MB Swap, which is 11% used).

    After restarting psql again I try:

    hopp=> drop table h;
    NOTICE:AbortTransaction and not in in-progress state
    NOTICE:AbortTransaction and not in in-progress state

    ????

    Not only that.

    It seems that the files h (the table) and h1 (the index) get deleted.
    (As it should)
    If I now recreate the table and index, I get a message that they already exist.
    (But the datafiles don't).

    If I do a \d, I see woth index and table, if I do a select * from h I get;
    WARN:cannot write block 1 of h1 [hopp] blind

    (Which is correct because the datafiles are missing).
    I got the same! You have to restart postmaster to get rid of
    durty blocks of h1 in buffer pool!
    I don't know why elog(FATAL) doesn't call something like
    abort() (as ASSERT() does) and simply call exit(0)!

    Vadim

    ------------------------------
  • Marc G. Fournier at Jun 12, 1997 at 4:25 pm

    On Fri, 13 Jun 1997, Vadim B. Mikheev wrote:

    Michael Reifenberger wrote:
    On Thu, 12 Jun 1997, Vadim B. Mikheev wrote:
    ...
    I've got FATAL!!!
    Trying to find cause ...
    Fine :-)

    So I'm not the only one who still sees a bug.
    Found and fixed!
    Someone forgot about aligning in indextuple.c:fastgetiattr().

    Also, COPY FROM was using char *index_nulls for index attrs null flags
    but (for unknown reasons) Datum idatum for index attrs values
    (instead of Datum *idatum).

    (I see that I had to check all code concerned indices while
    implementing multi-column ones ... I'm lazy).

    Well, I've got good results using Michael test data.
    So, 6.1 release shouldn't be postponed any more...
    (CVS changed)
    Michael...do you have a way of testing before midnight tonight? If not,
    please grab a copy at your earliest convience and confirm Vadim's results,
    *just in case*

    Unless I hear anything to the contrary, I will take the tarball generated
    tonight at midnight and release it during the day tomorrow...most likely
    shortly after 4:30pm, Atlantic Time


    Marc G. Fournier scrappy@hub.org
    Systems Administrator @ hub.org scrappy@freebsd.org

    ------------------------------
  • Vadim B. Mikheev at Jun 12, 1997 at 4:45 pm

    Michael Reifenberger wrote:

    On Thu, 12 Jun 1997, Vadim B. Mikheev wrote:
    ...
    I've got FATAL!!!
    Trying to find cause ...
    Fine :-)

    So I'm not the only one who still sees a bug.
    Found and fixed!
    Someone forgot about aligning in indextuple.c:fastgetiattr().

    Also, COPY FROM was using char *index_nulls for index attrs null flags
    but (for unknown reasons) Datum idatum for index attrs values
    (instead of Datum *idatum).

    (I see that I had to check all code concerned indices while
    implementing multi-column ones ... I'm lazy).

    Well, I've got good results using Michael test data.
    So, 6.1 release shouldn't be postponed any more...
    (CVS changed)

    Great thanks, Michael!

    Vadim

    ------------------------------
  • Michael Reifenberger at Jun 12, 1997 at 4:49 pm
    On Thu, 12 Jun 1997, Marc G. Fournier wrote:
    ...
    Michael...do you have a way of testing before midnight tonight? If not,
    please grab a copy at your earliest convience and confirm Vadim's results,
    *just in case*
    I test it now.

    I'll post my results soon after I got all testet.


    Bye!
    - ----
    Michael Reifenberger
    Plaut Software GmbH, R/3 Basis

    ------------------------------
  • Vadim B. Mikheev at Jun 12, 1997 at 5:08 pm

    Marc G. Fournier wrote:

    Michael...do you have a way of testing before midnight tonight? If not,
    please grab a copy at your earliest convience and confirm Vadim's results,
    *just in case*
    BTW, I'd post new code to Michael (just in case).
    Unless I hear anything to the contrary, I will take the tarball generated
    tonight at midnight and release it during the day tomorrow...most likely
    shortly after 4:30pm, Atlantic Time
    Vadim

    ------------------------------
  • Michael Reifenberger at Jun 12, 1997 at 6:17 pm
    ...
    Found and fixed!
    Someone forgot about aligning in indextuple.c:fastgetiattr().
    Indeed,
    It's fixed.

    You should now release 6.1 immediately before I start to find new bugs :-)

    Thanks all for your help.

    Bye!
    - ----
    Michael Reifenberger
    Plaut Software GmbH, R/3 Basis

    ------------------------------
  • Vadim B. Mikheev at Jun 12, 1997 at 7:04 pm

    Michael Reifenberger wrote:

    ...
    Found and fixed!
    Someone forgot about aligning in indextuple.c:fastgetiattr().
    Indeed,
    It's fixed.
    Fine news! We got there!
    You should now release 6.1 immediately before I start to find new bugs :-)
    We got rid of three bugs due to one your bug-report -
    your reports are welcome, at any time :-)

    Vadim

    ------------------------------

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouppgsql-hackers @
categoriespostgresql
postedJun 6, '97 at 7:19a
activeJun 12, '97 at 7:04p
posts31
users5
websitepostgresql.org...
irc#postgresql

People

Translate

site design / logo © 2021 Grokbase