Hello -hackers,

In the process of converting a multi-Tb datadabe from 8.2 to 8.3, Postgres
8.3 died at the failed assertion:

TRAP: FailedAssertion("!(((toast_pointer).va_extsize < (toast_pointer).va_rawsize - ((int32) sizeof(int32))))", File: "tuptoaster.c", Line: 1134)
LOG: server process (PID 8874) was terminated by signal 6: Aborted
LOG: terminating any other active server processes
LOG: all server processes terminated; reinitializing
LOG: database system was interrupted; last known up at 2008-02-20 07:43:00 MSK
LOG: database system was not properly shut down; automatic recovery in progress
LOG: redo starts at 78/BA00E060
LOG: record with zero length at 78/BC7A5FA8
LOG: redo done at 78/BC7A5F78
LOG: last completed transaction was at log time 2008-02-20 07:43:03.292665+03
LOG: autovacuum launcher started
LOG: database system is ready to accept connections

Unfortunately I cannot tell much more right now (I don't have
the exact name of the table even), although I suspect which column is that
(because I have only one column which is subject to toasting).

I was doing basically pg_dumpall_8.2 | psql_8.3
I can say that the tables are large ~ 100-500 Gb in size and contain the
column with the image in a special type "image", which is
toasted.
cas=# \d sdssdr5.frame
Table "sdssdr5.frame"
Column | Type | Modifiers
---------+------------------+-----------
fieldid | bigint | not null
..................
htmid | bigint | not null
img | image | not null

CREATE TYPE image (
INPUT = image_in,
OUTPUT = image_out,
INTERNALLENGTH = -1,
STORAGE = external
);

Although I may agree that it could be a problem occurring due to the bug
in type_in, type_out functions, I really doubt that, since it worked
perfectly with 8.1-8.2 and the code is quite simple.

I'll try to get the postmortem core dump, although it may require another
couple of days since the segfault occured after two days of dumping :(

Does anyone have ideas what could be the reason for the bug ?

Thanks in advance.

Regards,
Sergey

*******************************************************************
Sergey E. Koposov
Max Planck Institute for Astronomy/Cambridge Institute for Astronomy/Sternberg Astronomical Institute
Tel: +49-6221-528-349
Web: http://lnfm1.sai.msu.ru/~math
E-mail: math@sai.msu.ru

Search Discussions

  • Gregory Stark at Feb 20, 2008 at 11:04 am

    "Sergey E. Koposov" <math@sai.msu.ru> writes:

    In the process of converting a multi-Tb datadabe from 8.2 to 8.3, Postgres 8.3
    died at the failed assertion:

    TRAP: FailedAssertion("!(((toast_pointer).va_extsize < (toast_pointer).va_rawsize - ((int32) sizeof(int32))))", File: "tuptoaster.c", Line: 1134) ...
    Does anyone have ideas what could be the reason for the bug ?
    What the assert is saying is that the datum it's trying to toast is compressed
    but the compressed version is larger than the original -- which shouldn't ever
    happen because we don't store such data compressed.

    I haven't quite figured out where the error is yet though.

    --
    Gregory Stark
    EnterpriseDB http://www.enterprisedb.com
    Get trained by Bruce Momjian - ask me about EnterpriseDB's PostgreSQL training!
  • Teodor Sigaev at Feb 20, 2008 at 11:06 am
    Does anyone have ideas what could be the reason for the bug ?
    Compression of varlena's header, introduced in 8.3.

    --
    Teodor Sigaev E-mail: teodor@sigaev.ru
    WWW: http://www.sigaev.ru/
  • Gregory Stark at Feb 20, 2008 at 11:30 am

    "Sergey E. Koposov" <math@sai.msu.ru> writes:

    Hello -hackers,

    In the process of converting a multi-Tb datadabe from 8.2 to 8.3, Postgres 8.3
    died at the failed assertion:

    TRAP: FailedAssertion("!(((toast_pointer).va_extsize < (toast_pointer).va_rawsize - ((int32) sizeof(int32))))", File: "tuptoaster.c", Line: 1134)
    LOG: server process (PID 8874) was terminated by signal 6: Aborted
    LOG: terminating any other active server processes
    LOG: all server processes terminated; reinitializing
    LOG: database system was interrupted; last known up at 2008-02-20 07:43:00 MSK
    LOG: database system was not properly shut down; automatic recovery in progress
    LOG: redo starts at 78/BA00E060
    LOG: record with zero length at 78/BC7A5FA8
    LOG: redo done at 78/BC7A5F78
    LOG: last completed transaction was at log time 2008-02-20 07:43:03.292665+03
    LOG: autovacuum launcher started
    LOG: database system is ready to accept connections

    Unfortunately I cannot tell much more right now (I don't have the exact name of
    the table even), although I suspect which column is that (because I have only
    one column which is subject to toasting).

    I was doing basically pg_dumpall_8.2 | psql_8.3 I can say that the tables are
    large ~ 100-500 Gb in size and contain the column with the image in a special
    type "image", which is toasted. cas=# \d sdssdr5.frame
    Table "sdssdr5.frame"
    Column | Type | Modifiers
    ---------+------------------+-----------
    fieldid | bigint | not null
    ..................
    htmid | bigint | not null
    img | image | not null

    CREATE TYPE image (
    INPUT = image_in,
    OUTPUT = image_out,
    INTERNALLENGTH = -1,
    STORAGE = external
    );
    You aren't doing anything funny in the image_in function to generate
    compressed varlenas manually are you?

    Assuming not then it must be a case where we're saving less than 4 bytes and
    that's appearing as a saving in one place but then not somewhere else once you
    take into account the headers. Except I've just gone through the code looking
    for that kind of error and didn't spot it.

    I'll keep looking (or someone else will probably spot it before I do anyways)
    but if these images are mostly incompressible data you would probably be
    better off marking the columns as storage "external" so Postgres just toasts
    them as-is instead of trying to compress them first with:

    ALTER column SET STORAGE EXTERNAL

    --
    Gregory Stark
    EnterpriseDB http://www.enterprisedb.com
    Ask me about EnterpriseDB's On-Demand Production Tuning
  • Gregory Stark at Feb 20, 2008 at 11:55 am

    "Gregory Stark" <stark@enterprisedb.com> writes:


    CREATE TYPE image (
    INPUT = image_in,
    OUTPUT = image_out,
    INTERNALLENGTH = -1,
    STORAGE = external
    );
    ALTER column SET STORAGE EXTERNAL
    Hum. I just noticed that you had set STORAGE = external in your type
    declaration. That makes it pretty odd that it would be going through this code
    path at all.

    Could you send

    select *
    from pg_attribute
    where attrelid = 'sdssdr5.frame'::regclass
    and attname = 'img';

    --
    Gregory Stark
    EnterpriseDB http://www.enterprisedb.com
    Ask me about EnterpriseDB's On-Demand Production Tuning
  • Sergey E. Koposov at Feb 20, 2008 at 12:19 pm

    On Wed, 20 Feb 2008, Gregory Stark wrote:
    Could you send

    select *
    from pg_attribute
    where attrelid = 'sdssdr5.frame'::regclass
    and attname = 'img';
    cas=# select *
    from pg_attribute
    where attrelid = 'sdssdr5.frame'::regclass
    and attname = 'img';
    attrelid | attname | atttypid | attstattarget | attlen | attnum | attndims | attcacheoff | atttypmod | attbyval | attstorage | attalign | attnotnull | atthasdef | attisdropped | attislocal | attinhcount
    ----------+---------+----------+---------------+--------+--------+----------+-------------+-----------+----------+------------+----------+------------+-----------+--------------+------------+-------------
    16856 | img | 16418 | -1 | -1 | 29 | 0 | -1 | -1 | f | e | i | t | f | f | t | 0
    (1 row)

    Regards,
    Sergey

    *******************************************************************
    Sergey E. Koposov
    Max Planck Institute for Astronomy/Cambridge Institute for Astronomy/Sternberg Astronomical Institute
    Tel: +49-6221-528-349
    Web: http://lnfm1.sai.msu.ru/~math
    E-mail: math@sai.msu.ru
  • Sergey E. Koposov at Feb 20, 2008 at 12:17 pm

    On Wed, 20 Feb 2008, Gregory Stark wrote:

    You aren't doing anything funny in the image_in function to generate
    compressed varlenas manually are you?
    No, I don't. The only thing I do there is
    unsigned char *in = PG_GETARG_CSTRING(0);//"AABBCCDDEE1122";//
    and return the pointer to the palloced constructed standard varlena
    datum with

    typedef struct
    {
    int4 length;
    unsigned char data[1];
    } image;

    image *im = (image *) palloc(VARHDRSZ + out_len);
    memset(im, 0, VARHDRSZ + out_len);
    im->length = out_len + VARHDRSZ;
    /* fill the im->data
    .......
    */
    PG_RETURN_POINTER(im);

    Regards,
    Sergey

    *******************************************************************
    Sergey E. Koposov
    Max Planck Institute for Astronomy/Cambridge Institute for Astronomy/Sternberg Astronomical Institute
    Tel: +49-6221-528-349
    Web: http://lnfm1.sai.msu.ru/~math
    E-mail: math@sai.msu.ru
  • Gregory Stark at Feb 20, 2008 at 12:47 pm

    "Sergey E. Koposov" <math@sai.msu.ru> writes:

    typedef struct
    {
    int4 length;
    unsigned char data[1];
    } image;

    image *im = (image *) palloc(VARHDRSZ + out_len);
    memset(im, 0, VARHDRSZ + out_len);
    im->length = out_len + VARHDRSZ;
    Ah, that's not going to work in 8.3 any longer. You have to change this to:

    SET_VARSIZE(im, out_len+VARHDRSZ)

    And you have to access the length with VARSIZE_ANY_EXHDR() (or a few other
    macros but that's the most convenient).

    Phew. You had me scared there.

    --
    Gregory Stark
    EnterpriseDB http://www.enterprisedb.com
    Ask me about EnterpriseDB's Slony Replication support!
  • Sergey E. Koposov at Feb 20, 2008 at 1:06 pm

    On Wed, 20 Feb 2008, Gregory Stark wrote:
    Ah, that's not going to work in 8.3 any longer. You have to change this to:

    SET_VARSIZE(im, out_len+VARHDRSZ)

    Phew. You had me scared there.
    Thank you. Sorry for scaring :)
    I hope that everything will work fine now.

    Regards,
    Sergey

    *******************************************************************
    Sergey E. Koposov
    Max Planck Institute for Astronomy/Cambridge Institute for Astronomy/Sternberg Astronomical Institute
    Tel: +49-6221-528-349
    Web: http://lnfm1.sai.msu.ru/~math
    E-mail: math@sai.msu.ru

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouppgsql-hackers @
categoriespostgresql
postedFeb 20, '08 at 10:39a
activeFeb 20, '08 at 1:06p
posts9
users3
websitepostgresql.org...
irc#postgresql

People

Translate

site design / logo © 2022 Grokbase