Hi.

Trying to implement some simple digest routines via UDFs and for whatever
reason I get: ERROR: invalid memory alloc request size 4294967293 on
PG_RETURN_TEXT_P(); any ideas what the issue is exactly?

The code is verified as working when pulled out of a UDF and put into a
normal C program. Thanks.

dp_blag=# CREATE OR REPLACE FUNCTION sha512(text) RETURNS text AS
'dp_sha512.so','dp_sha512' LANGUAGE C STRICT;
CREATE FUNCTION
dp_blag=# SELECT sha512('jf');
ERROR: invalid memory alloc request size 4294967293

PG_FUNCTION_INFO_V1(dp_sha512);

Datum
dp_sha512(PG_FUNCTION_ARGS)
{
text * hash;
text * plain;
int32 hlen;

plain = (text *)PG_GETARG_TEXT_P(0);

if (NULL == plain) {
ereport(ERROR,
(errcode(ERRCODE_NULL_VALUE_NOT_ALLOWED),
errmsg("invalid argument to function"))
);
PG_RETURN_NULL();
}

hlen = SHA512_DIGEST_LENGTH;

hash = (text *)palloc(hlen+1);

if (NULL == hash) {
ereport(ERROR,
(errcode(ERRCODE_INTERNAL_ERROR),
errmsg("palloc() failed"))
);
PG_RETURN_NULL();
}

memset(VARDATA(hash), 0, hlen);
SHA512(VARDATA(plain), hlen, VARDATA(hash));
PG_RETURN_TEXT_P(hash);
}

Search Discussions

  • Pavel Stehule at Aug 13, 2007 at 8:39 am

    hash = (text *)palloc(hlen+1);
    ??? palloc(hlen + VARHDRSZ)
    memset(VARDATA(hash), 0, hlen);
    SHA512(VARDATA(plain), hlen, VARDATA(hash));

    ++ VARATT_SIZEP (hash) = VARHDRSZ + ....;
    PG_RETURN_TEXT_P(hash);
    }

    ---------------------------(end of broadcast)---------------------------
    TIP 3: Have you checked our extensive FAQ?

    http://www.postgresql.org/docs/faq
  • Martijn van Oosterhout at Aug 13, 2007 at 8:43 am

    On Mon, Aug 13, 2007 at 03:17:36PM +0000, jf wrote:
    Hi.

    Trying to implement some simple digest routines via UDFs and for whatever
    reason I get: ERROR: invalid memory alloc request size 4294967293 on
    PG_RETURN_TEXT_P(); any ideas what the issue is exactly?
    A few points about your code:
    - The tests against NULL are useless: palloc never returns NULL and
    the argument to the function won't be NULL either (it's decalred
    STRICT).
    - Your palloc should be for more, the header may be more than one byte
    VAR_HEADER_LEN or some such would be more appropriate)
    - Finally, what is probably the actual problem, at no point did you
    assign a length to the hash variable, ie VARLEN(hash)=foo.

    Hope this helps,
    --
    Martijn van Oosterhout <kleptog@svana.org> http://svana.org/kleptog/
    From each according to his ability. To each according to his ability to litigate.
  • Jf at Aug 13, 2007 at 9:22 am
    Yup, an uninitialized variable makes sense, That appears to have worked, I appreciate it. In regards to the checks for
    NULL, I wasn't sure if it was necessary or not so I opted for the safe
    route, thanks for clearing that up for me.

    The output from the function is binary, does it matter if I use text or
    bytea? Finally, how do I cast in SQL? when I try to pass the output to
    encode() it says it can't find a function taking those types.

    Thanks a lot!
    On Mon, 13 Aug 2007, Martijn van Oosterhout wrote:

    Date: Mon, 13 Aug 2007 10:42:52 +0200
    From: Martijn van Oosterhout <kleptog@svana.org>
    To: jf <jf@danglingpointers.net>
    Cc: pgsql-general@postgresql.org
    Subject: Re: [GENERAL] UDFs
    On Mon, Aug 13, 2007 at 03:17:36PM +0000, jf wrote:
    Hi.

    Trying to implement some simple digest routines via UDFs and for whatever
    reason I get: ERROR: invalid memory alloc request size 4294967293 on
    PG_RETURN_TEXT_P(); any ideas what the issue is exactly?
    A few points about your code:
    - The tests against NULL are useless: palloc never returns NULL and
    the argument to the function won't be NULL either (it's decalred
    STRICT).
    - Your palloc should be for more, the header may be more than one byte
    VAR_HEADER_LEN or some such would be more appropriate)
    - Finally, what is probably the actual problem, at no point did you
    assign a length to the hash variable, ie VARLEN(hash)=foo.

    Hope this helps,
  • Hubert depesz lubaczewski at Aug 13, 2007 at 9:30 am

    On Mon, Aug 13, 2007 at 03:17:36PM +0000, jf wrote:
    Trying to implement some simple digest routines via UDFs and for whatever
    reason I get: ERROR: invalid memory alloc request size 4294967293 on
    PG_RETURN_TEXT_P(); any ideas what the issue is exactly?
    just checking - you do realize that it's reinventing the wheel?
    i mean - the code already exists, and is bundled in standard postgresql
    sources?
    if not - use pgcrypto module from contrib/

    depesz

    --
    quicksil1er: "postgres is excellent, but like any DB it requires a
    highly paid DBA. here's my CV!" :)
    http://www.depesz.com/ - blog dla ciebie (i moje CV)
  • Jf at Aug 13, 2007 at 9:34 am
    my understanding was that pgcrypto was not compiled by default?
    Furthermore, finding next to no documentation about it online and deciding
    I only needed one function instead of an entire crypto API i decided it
    would make the most sense to just code the 10 lines to do it myself.

    On Mon, 13 Aug 2007, hubert depesz lubaczewski wrote:

    Date: Mon, 13 Aug 2007 11:29:58 +0200
    From: hubert depesz lubaczewski <depesz@depesz.com>
    To: jf <jf@danglingpointers.net>
    Cc: pgsql-general@postgresql.org
    Subject: Re: [GENERAL] UDFs
    On Mon, Aug 13, 2007 at 03:17:36PM +0000, jf wrote:
    Trying to implement some simple digest routines via UDFs and for whatever
    reason I get: ERROR: invalid memory alloc request size 4294967293 on
    PG_RETURN_TEXT_P(); any ideas what the issue is exactly?
    just checking - you do realize that it's reinventing the wheel?
    i mean - the code already exists, and is bundled in standard postgresql
    sources?
    if not - use pgcrypto module from contrib/

    depesz
  • Hubert depesz lubaczewski at Aug 13, 2007 at 11:23 am

    On Mon, Aug 13, 2007 at 05:07:50PM +0000, jf wrote:
    my understanding was that pgcrypto was not compiled by default?
    your function is also not compiled on default. but pgcrypto is at the
    very least available by default (in sources, or in precompiled
    packages).
    Furthermore, finding next to no documentation about it online and deciding
    i dont know about online documentation, but in installed pgcrypto you
    have a nice readme with all functions described.
    I only needed one function instead of an entire crypto API i decided it
    would make the most sense to just code the 10 lines to do it myself.
    i'm not opposing to the idea. i was just checking if you do know about
    it :)

    best regards,

    depesz

    --
    quicksil1er: "postgres is excellent, but like any DB it requires a
    highly paid DBA. here's my CV!" :)
    http://www.depesz.com/ - blog dla ciebie (i moje CV)
  • Jf at Aug 13, 2007 at 11:48 am

    your function is also not compiled on default. but pgcrypto is at the
    very least available by default (in sources, or in precompiled
    packages).
    Yes I understand, and trust me I'm typically not a 'not made in my house'
    type, but google for pgcrypto, you don't find much out there other than
    'look in contrib/', and honestly I wasn't about to sit down and recompile
    everything to get a bunch of functionality I really didn't need, I (in
    hindsight incorrectly) decided it would be quicker just to write a UDF.

    Where the incorrectly part is that I've been battling for a couple hours
    trying to figure out why even though I had changed the UDF to use BYTEA
    primitives instead of TEXT it was still using TEXTs, and I just learned
    something new, CREATE OR REPLACE FUNCTION [...] doesn't replace the
    function if you change the types; it makes sense, I've just trying to
    figure out wth is going on.
    i dont know about online documentation, but in installed pgcrypto you
    have a nice readme with all functions described.
    Expecting people who may or may not have the source sitting around anymore
    to download the source to look at the documentation to even determine if
    they want to use that code is not a very effective method of marketing,
    imho of course.

    This holds especially true when the online manual for the
    DB itself is lacking in many places (ie can palloc() fail? does palloc0()
    zero out the memory returned? et cetera), I had to dig through the header
    files just to find the various types and such, and then its still not
    entirely clear what exactly the header for the variable length types is,
    in one header file it gets typedef'd to a structure defined right above it
    that contains a length field and an array of a single character (does that
    serve as a pointer to the beginning of the data?), but then the
    VARDATA()/VARSIZE()/et cetera macro's appear to work on an entirely
    different structure, but there is no immediate correlation between the two
    structures in the headers.

    It's a bit sad because the documentation for pgcrypto is actually quite a
    bit better than for the UDFs, and it wasn't until after I had made the
    decision that the UDFs would be easier and fought with it for some time
    that I realized this.

    i'm not opposing to the idea. i was just checking if you do know about
    it :)
    understood, I appreciate the suggestion. In addition I couldn't find any
    documentation that told me how to install the functions in pgcrypto (do I
    need to CREATE FUNCTION for every function in there?), the README mentions
    a .sql file thats supposed to get you up and going, but the only .sql file
    I found in the directory was an uninstall .sql file, am I expected to grok
    it and do the inverse for every step? et cetera.

    I don't mean to come off as a complainer, but the documentation for the DB
    overall seems to be lacking, and that was my main reason for opting out,
    even if now that all is said and done it would've been easier to use
    pgcrypto.
    best regards,

    depesz

    cheers,

    jf
  • Hubert depesz lubaczewski at Aug 13, 2007 at 11:56 am

    On Mon, Aug 13, 2007 at 07:22:26PM +0000, jf wrote:
    understood, I appreciate the suggestion. In addition I couldn't find any
    documentation that told me how to install the functions in pgcrypto (do I
    need to CREATE FUNCTION for every function in there?), the README mentions
    a .sql file thats supposed to get you up and going, but the only .sql file
    I found in the directory was an uninstall .sql file, am I expected to grok
    it and do the inverse for every step? et cetera.
    1. as for installing contrib - usually when you install psotgresql from
    prebuilt binary packages, there is also contrib package. for example on
    ubuntu it is postgresql-contrib-8.2, so simple: apt-get install
    postgresql-contrib-8.2 will install it.

    as for installation in sql - for every contrib module you get sql
    script. it is usually called <contrib_module>.sql (pgcrypto.sql).

    usually it's in: <PREFIX>/share/postgresql/contrib/pgcrypto.sql

    in the database you want to use pgcrypto functions, you simply run this
    sql (as superuser), and that's all.

    depesz

    --
    quicksil1er: "postgres is excellent, but like any DB it requires a
    highly paid DBA. here's my CV!" :)
    http://www.depesz.com/ - blog dla ciebie (i moje CV)
  • Jf at Aug 13, 2007 at 12:02 pm

    1. as for installing contrib - usually when you install psotgresql from
    prebuilt binary packages, there is also contrib package. for example on
    ubuntu it is postgresql-contrib-8.2, so simple: apt-get install
    postgresql-contrib-8.2 will install it.
    I actually built from source, and just didnt know that crypto wasnt
    included until I was already up and running and had gotten far enough into
    things that I didn't want to restart.
    as for installation in sql - for every contrib module you get sql
    script. it is usually called <contrib_module>.sql (pgcrypto.sql).

    usually it's in: <PREFIX>/share/postgresql/contrib/pgcrypto.sql
    in the database you want to use pgcrypto functions, you simply run this
    sql (as superuser), and that's all.
    theory# pwd
    /home/jf/postgresql-8.2.4
    theory# cd share
    bash: cd: share: No such file or directory
    theory# pwd
    /home/jf/postgresql-8.2.4
    theory# ls share
    ls: cannot access share: No such file or directory
    theory# find ./ -name pgcrypto.sql
    theory# cd contrib/pgcrypto/
    theory# ls *.sql
    uninstall_pgcrypto.sql
    theory# ls
    Makefile crypt-blowfish.c expected imath.h mbuf.h
    pgcrypto.c pgp-cfb.c pgp-info.c pgp-pgsql.c
    pgp-s2k.c px-crypt.h random.c sha1.c sql
    README.pgcrypto crypt-des.c fortuna.c internal-sha2.c md5.c
    pgcrypto.h pgp-compress.c pgp-mpi-internal.c pgp-pubdec.c pgp.c
    px-hmac.c rijndael.c sha1.h uninstall_pgcrypto.sql
    blf.c crypt-gensalt.c fortuna.h internal.c md5.h
    pgcrypto.sql.in pgp-decrypt.c pgp-mpi-openssl.c pgp-pubenc.c pgp.h
    px.c rijndael.h sha2.c
    blf.h crypt-md5.c imath.c mbuf.c openssl.c
    pgp-armor.c pgp-encrypt.c pgp-mpi.c pgp-pubkey.c
    px-crypt.c px.h rijndael.tbl sha2.h
  • Hubert depesz lubaczewski at Aug 13, 2007 at 12:11 pm

    On Mon, Aug 13, 2007 at 07:36:41PM +0000, jf wrote:
    usually it's in: <PREFIX>/share/postgresql/contrib/pgcrypto.sql
    in the database you want to use pgcrypto functions, you simply run this
    sql (as superuser), and that's all.
    theory# pwd
    /home/jf/postgresql-8.2.4
    theory# cd share
    bash: cd: share: No such file or directory
    theory# pwd
    /home/jf/postgresql-8.2.4
    theory# ls share
    ls: cannot access share: No such file or directory
    share of *installed* system. if you compiled with --prefix=/usr/local,
    then it would be /usr/local/share/postgresql/...
    theory# find ./ -name pgcrypto.sql
    theory# cd contrib/pgcrypto/
    theory# ls *.sql
    if you didn't compile it - it is not built.
    simply:
    cd contrib/pgcrypto
    make
    make install
    and you should be fine.

    i usually compile postgresql in this way:
    cd postgre*
    ./configure --.... some options
    make
    make install
    cd contrib
    make
    make install

    in this way i have all contrib modules compiled.

    of course - you dont need all (on the other hand - i strongly suggest
    that you get some familiarity with it, as there are some real gems).

    depesz

    --
    quicksil1er: "postgres is excellent, but like any DB it requires a
    highly paid DBA. here's my CV!" :)
    http://www.depesz.com/ - blog dla ciebie (i moje CV)
  • Jf at Aug 13, 2007 at 12:14 pm

    share of *installed* system. if you compiled with --prefix=/usr/local,
    then it would be /usr/local/share/postgresql/...
    Ah you have to forgive me, I'm in the states and its quite late ;]
    of course - you dont need all (on the other hand - i strongly suggest
    that you get some familiarity with it, as there are some real gems).
    Well as I continue working on my backend and find that I need some other
    functionality, I'll check there first ;]

    Thanks again.
  • Martijn van Oosterhout at Aug 13, 2007 at 12:14 pm

    On Mon, Aug 13, 2007 at 07:36:41PM +0000, jf wrote:
    usually it's in: <PREFIX>/share/postgresql/contrib/pgcrypto.sql
    in the database you want to use pgcrypto functions, you simply run this
    sql (as superuser), and that's all.
    theory# pwd
    /home/jf/postgresql-8.2.4
    theory# cd share
    bash: cd: share: No such file or directory
    He means the INSTALLPREFIX, which is usually /usr or /usr/local. Your
    problem is that you havn't build it yet (run make) which will generate
    the actual module and you need to install it to get the scripts into
    the right locations.

    If you can, I'd suggest installing binary versions of the contrib
    modules (it's called postgresql-8.2-contrib in debian for example).
    That saves you having to worry about sources, paths, compilations, etc).

    Have a nice day,
    --
    Martijn van Oosterhout <kleptog@svana.org> http://svana.org/kleptog/
    From each according to his ability. To each according to his ability to litigate.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouppgsql-general @
categoriespostgresql
postedAug 13, '07 at 8:11a
activeAug 13, '07 at 12:14p
posts13
users4
websitepostgresql.org
irc#postgresql

People

Translate

site design / logo © 2022 Grokbase